Cloudera promises 'Google-like' Big Data dream in minutes

Hadoop shop automates so you don't have to


Updated Cloudera has delivered a "substantial" update to its open source Hadoop distribution.

On Wednesday, Cloudera rolled out Cloudera Enterprise 3.5, two months after shipping a major upgrade to its Hadoop distribution called Cloudera Distribution of Apache Hadoop (CDH) 3.0.

Whereas CDH 3.0 expanded Cloudera's Hadoop stack from three components to 10, the idea behind Cloudera Enterprise 3.5 is to make that distro easier to manage and deploy for IT shops outside the ranks of Hadoop's super-users like Facebook, Yahoo!, and LinkedIn. Cloudera provides services and support for such mainstream users.

The changes will let you install and configure a full Google-like infrastructure "in a couple of minutes", product vice president Charles Zedlewski told The Register. "We have done a substantial update." Hadoop is based on Google's GFS and MapReduce platforms.

Zedlewski said Cloudera Enterprise 3.5 automates configuration changes, service restarts, and the addition and removal of hardware. There's also an Activity Monitor that consolidates user activity across components to provide both a real-time and historical view of user activities and jobs.

"[We have] expanded the capability of the management suite from monitoring and discovery of issues, to diagnostic problems, to automating changes, and setting long-term changes," Zedlewski said.

Cloudera has also enhanced the Hadoop Resource and Authorization Manager, facilitating rollbacks and improving security with LDAP systems.

Hadoop is an architecture for crunching huge amounts of data using a network of distributed servers. Nutch web crawler creator Doug Cutting based the platform on research papers describing Google GFS and MapReduce., and it is now an Apache Software Foundation (ASF) project.

Today, Cloudera also released a free "Express" edition of its Service and Configuration Manager module used in Cloudera Enterprise 3.5 that will automate the installation and configuration of Hadoop on a cluster of up to 50 nodes. Meanwhile, the company has also donated code for its packaging and testing suite to ASF, under a project called Bigtop. The idea is to help improve packaging and interoperability testing for Hadoop and related modules.

Among Bigtop's initial committers is Canonical, chief commercial steward of Ubuntu. Cloudera has supported packaging for Ubuntu Linux for the last two-and-a-half years.

Zedlewski said Cloudera will add a further three or four modules to the current stack, among them a compression algorithm that leverages Google's Snappy to speed up data import and export.

This will be added in an update to CDH in the next month or so, Zedlewski said. Other components are due in "the CDH 4 timeframe", he said, while Cloudera is also looking at enhancements around high-availability features in the core Hadoop module. Zedlewski would not provide a date for CDH 4, but he said "work is well underway". ®

This article has been updated to clarify Cloudera has delivered Cloudera Enterprise 3.5.


Other stories you might like

  • Google has more reasons why it doesn't like antitrust law that affects Google
    It'll ruin Gmail, claims web ads giant

    Google has a fresh list of reasons why it opposes tech antitrust legislation making its way through Congress but, like others who've expressed discontent, the ad giant's complaints leave out mention of portions of the proposed law that address said gripes.

    The law bill in question is S.2992, the Senate version of the American Innovation and Choice Online Act (AICOA), which is closer than ever to getting votes in the House and Senate, which could see it advanced to President Biden's desk.

    AICOA prohibits tech companies above a certain size from favoring their own products and services over their competitors. It applies to businesses considered "critical trading partners," meaning the company controls access to a platform through which business users reach their customers. Google, Apple, Amazon, and Meta in one way or another seemingly fall under the scope of this US legislation. 

    Continue reading
  • Makers of ad blockers and browser privacy extensions fear the end is near
    Overhaul of Chrome add-ons set for January, Google says it's for all our own good

    Special report Seven months from now, assuming all goes as planned, Google Chrome will drop support for its legacy extension platform, known as Manifest v2 (Mv2). This is significant if you use a browser extension to, for instance, filter out certain kinds of content and safeguard your privacy.

    Google's Chrome Web Store is supposed to stop accepting Mv2 extension submissions sometime this month. As of January 2023, Chrome will stop running extensions created using Mv2, with limited exceptions for enterprise versions of Chrome operating under corporate policy. And by June 2023, even enterprise versions of Chrome will prevent Mv2 extensions from running.

    The anticipated result will be fewer extensions and less innovation, according to several extension developers.

    Continue reading
  • I was fired for blowing the whistle on cult's status in Google unit, says contractor
    The internet giant, a doomsday religious sect, and a lawsuit in Silicon Valley

    A former Google video producer has sued the internet giant alleging he was unfairly fired for blowing the whistle on a religious sect that had all but taken over his business unit. 

    The lawsuit demands a jury trial and financial restitution for "religious discrimination, wrongful termination, retaliation and related causes of action." It alleges Peter Lubbers, director of the Google Developer Studio (GDS) film group in which 34-year-old plaintiff Kevin Lloyd worked, is not only a member of The Fellowship of Friends, the exec was influential in growing the studio into a team that, in essence, funneled money back to the fellowship.

    In his complaint [PDF], filed in a California Superior Court in Silicon Valley, Lloyd lays down a case that he was fired for expressing concerns over the fellowship's influence at Google, specifically in the GDS. When these concerns were reported to a manager, Lloyd was told to drop the issue or risk losing his job, it is claimed. 

    Continue reading
  • Google recasts Anthos with hitch to AWS Outposts
    If at first you don't succeed, change names and try again

    Google Cloud's Anthos on-prem platform is getting a new home under the search giant’s recently announced Google Distributed Cloud (GDC) portfolio, where it will live on as a software-based competitor to AWS Outposts and Microsoft Azure Stack.

    Introduced last fall, GDC enables customers to deploy managed servers and software in private datacenters and at communication service provider or on the edge.

    Its latest update sees Google reposition Anthos on-prem, introduced back in 2020, as the bring-your-own-server edition of GDC. Using the service, customers can extend Google Cloud-style management and services to applications running on-prem.

    Continue reading
  • Google offers $118m to settle gender discrimination lawsuit
    Don't even think about putting LaMDA on the compensation committee

    Google has promised to cough up $118 million to settle a years-long gender-discrimination class-action lawsuit that alleged the internet giant unfairly pays men more than women.

    The case, launched in 2017, was led by three women, Kelly Ellis, Holly Pease, and Kelli Wisuri, who filed a complaint alleging the search giant hires women in lower-paying positions compared to men despite them having the same qualifications. Female staff are also less likely to get promoted, it was claimed.

    Gender discrimination also exists within the same job tier, too, the complaint stated. Google was accused of paying women less than their male counterparts despite them doing the same work. The lawsuit was later upgraded to a class-action status when a fourth woman, Heidi Lamar, joined as a plaintiff. The class is said to cover more than 15,000 people.

    Continue reading
  • Ubuntu Touch OTA-23 is coming: Do you have one of the older model phones that can test it?
    Linux fondleslab effort continues to update 16.04-based phone/tablet distro

    The UBPorts community is in the final stages of preparing its next release and it's calling for testers.

    OTA-23 is getting close – the project's Github kanban looks quite good to us – and if you're lucky enough to have one of the project's supported devices lying around, then you can help.

    Many of them are a few years old now, so there's a good chance that you've already replaced them and they sit unloved and neglected in a drawer. The starred entries in the list of devices are the best supported and should have no show-stopping problems. In order of seniority, that means: the LG-made Google Nexus 5 (2013); the original Oneplus One (2014); two models of Sony Xperia X, the F5121 and F5122 (2016); and Google's Pixel 3a and 3a XL (2019).

    Continue reading
  • Google: How we tackled this iPhone, Android spyware
    Watching people's every move and collecting their info – not on our watch, says web ads giant

    Spyware developed by Italian firm RCS Labs was used to target cellphones in Italy and Kazakhstan — in some cases with an assist from the victims' cellular network providers, according to Google's Threat Analysis Group (TAG).

    RCS Labs customers include law-enforcement agencies worldwide, according to the vendor's website. It's one of more than 30 outfits Google researchers are tracking that sell exploits or surveillance capabilities to government-backed groups. And we're told this particular spyware runs on both iOS and Android phones.

    We understand this particular campaign of espionage involving RCS's spyware was documented last week by Lookout, which dubbed the toolkit "Hermit." We're told it is potentially capable of spying on the victims' chat apps, camera and microphone, contacts book and calendars, browser, and clipboard, and beam that info back to base. It's said that Italian authorities have used this tool in tackling corruption cases, and the Kazakh government has had its hands on it, too.

    Continue reading

Biting the hand that feeds IT © 1998–2022