Accel ponies up $100m to juice big data biz

Using the elephant to mask the elephant

Hadoop World We have entered the business era of the big data eon. That's just a fancy way of saying that software companies are trying to take Hadoop and the myriad new tools that wrap around it and convert them into money by unseating incumbents in data warehousing and analytics – and, quite frankly, by doing some jobs that no one could handle before.

Now the big money is starting to move in: venture capital behemoth Accel Partners is ponying up $100m to invest in big-data startups that are looking to become the application giants that will make a killing by creating Hadoop-based applications that are fit for normal business consumption.

Ping Li, a partner at Accel since 2004, said in his keynote address at the Hadoop World Conference in New York on Tuesday that the opportunity for making money was enormous. The volume of data being kept is growing very quickly, and every kind of event log or twitch of a company's applications and web servers is being preserved in the hope that it can be mined for insight into customers and markets.

Moreover, unstructured data types are proliferating and companies don't want to rely in the old batch-oriented ways of current Hadoop setups to chew through this data. As usual, business managers want real-time data, just as they did in the 1970s when online transaction processing replaced batch operations for core applications.

This trend is going to mean investment flowing to companies that can improve the Hadoop infrastructure. And unlike those early OLTP days, companies want applications to ride on top of Hadoop and its various adjuncts to provide them with distilled information, and they don't want to build those apps from scratch.

"These new types of data are breaking legacy platforms," Li said. "And this is a big market, not just something for Web 2.0 companies." And to that end, Accel is putting up $100m for its Big Data Fund to help find the Microsofts and Facebooks of the Hadoop application era – and, of course, hopefully before the competition does.

"If you are an entrepreneur with a great idea, I implore you, take some money from this guy," said Cloudera CEO Mike Olson, who also spoke during the keynote. Accel is one of the early investors in Cloudera, which was first out the door with a commercially supported release of the Apache Hadoop stack. "We've got a chance to do something that only comes along once in a generation."

Yahoo! created Hadoop to mimic the operations of Google's search engine circa 2001. It is written in Java and consists of a MapReduce framework that spreads log files, web click streams, and other unstructured data across a network of servers. Then a MapReduce algorithm looks for connections between all this data, as well as any structured data ripped out of database and pumped into the Hadoop Distributed File System.

The mapping part of the algorithm means telling calculations where they need to go to find the pieces of data they need, and the reducing part refers to bringing together the results across thousands of nodes to pull together a summarized answer.

Yahoo! open sourced Hadoop in 2009, and creator Doug Cutting named it after his kid's stuffed elephant. A few Yahoo! employees left to start Cloudera, which jumped out there first to commercialize Hadoop. Since then, the remaining Yahoo! team has been spun out to create HortonWorks, IBM has commercialized its own BigInsights variant, and a number of players including Parascale, MapR, and Appistry have all come out with alternative files systems to HDFS. Platform Computing has even gone so far as to take its own Java-based financial messaging system used for risk analysis and high speed trading, and make it compatible with Hadoop APIs as an alternative to Hadoop itself.

There's a lot going on out there in Hadoop World. That is just the tip of the elephant's trunk.

Accel was an early investor in Cloudera, which announced $40m in Series D funding at the conference this morning, which it is hosting. This time around, Ignition Partners lead the investment with Accel Partners, Greylock Partners, Meritech Capital Partners, and In-Q-Tel also kicked in some cash to Cloudera. The company has raised $76m in four rounds of funding in the past three years, and Accel has been in all four rounds.

Li says that big-data platforms such as Hadoop – there are others, but they don't generate the same level of excitement – are going mainstream, and that "native" big data applications and services will quickly emerge once companies get Hadoop installed and learn how to use it.

In the prior computing era, where data and systems were not so big – say from 1980 to 2010, in Li's oversimplification – physical mainframes, midrange, and x86 servers were the infrastructure, and applications ran on relational database management systems.

This time around, the infrastructure is going to be virtualized private and public clouds. Hadoop and its add-ons will comprise the data management layer, and a whole new set of analytics, customer relationship management, collaboration, security, and vertical applications aimed at financial services, healthcare, and others areas are going to ride that elephant.

"Multi-billion dollar software companies are going to be created," said Li – and Accel wants to get in on the action.

But before this can happen, Hadoop has had to do a lot of growing up. Hadoop is "a complex piece of multi-tenant infrastructure and it has to behave in your data center," explained Olson.

"It's not enough to have a platform that sophisticated Java developers can use. Business users need to access this," he said.

And over the next year, Olson expects all kinds of apps to come to market to make it easier for regular people – not just Java wizards – to make use of Hadoop. And, if it is done right, they won't even know they're using Hadoop. ®

Similar topics

Other stories you might like

  • Mastering metadata key to shifting data fast, says Arcitecta
    A new transmission protocol can work lightning fast, but only with very thorough records to pull from

    Companies that move and analyze huge volumes of data are always on the lookout for faster ways to do it. One Australian company says it has created a protocol that can "transmit terabytes per minute across the globe."

    The company, Arcitecta, which has an almost 25-year history, has just announced the new Livewire protocol, which is part of their data management platform, Mediaflux, used by institutions including the Australian Department of Defense, drug maker Novartis, and the Dana Farber Cancer Institute.

    According to CEO Jason Lohrey, Livewire itself has already made an impact for some of the largest data movers. "One of our customers transmits petabytes of data around the globe, he told The Register.

    Continue reading
  • Real-time data analytics firm Tinybird nets $37m in Series A
    Millions of rows per second in real time, so the legend goes...

    A data analytics company claiming to be able to process millions of rows per second, in real time, has just closed out a Series A funding round to take-in $37 million.

    Tinybird raised the funds via investors Crane Ventures, Datadog CPO Amit Agarwal, and Vercel CEO Guillermo Rauch, along with new investments from CRV and Singular Ventures.

    Tinybird's Stephane Cornille, said the company plans to use the funds to expand into new regions, build connectors for additional cloud providers, create new ways for users to build their own connectors, provide tighter Git integration, Schema management and migrations, and better defaults and easier materialization.

    Continue reading
  • Big data means big money for the UK government as £2bn tender mooted
    Central procurement team tickles the market with tantalising offer... but what for?

    The UK government is putting the feelers out for a bundle of big data products and services in a move that could kick off £2bn in tendering.

    Cabinet Office-run Crown Commercial Service (CCS), which sets up procurement on behalf of central government ministries and other public sector organisations, has published an early market engagement to test the suppliers' interest in a framework for so-called big data and analytics systems.

    According to the prior information notice: "Big data and analytics is an emerging and evolving capability, with its prominence heightened by COVID. It is fast becoming recognised as business critical and a core business function, with many government departments now including chief data officers. The National Data Strategy and implementation of its missions reinforce the requirement to access and interrogate Government data more effectively to improve public services."

    Continue reading

Biting the hand that feeds IT © 1998–2022