Blink and you'll miss it. Big Data analytics is a huge focus of startup activity with young and new companies getting a green light from blue-chip customers and VCs almost all the time.
Well, that’s an exaggeration, but take your eye off the ball in this area and the landscape can change in fairly short order. Elasticsearch, which provides a document-oriented, open source big data search and analytics tool, didn't exist 24 months ago, yet its total funding is now $104m and there have been 11.4m downloads of its software.
Oh, and Facebook is one of its customers. So are the Daily Mail and Guardian newspapers, as well as TomTom, and EMC*.
One bank customer has 700 elastic search nodes with nine different use cases, each started separately. The product's use grows almost virally as individuals with data analysis problems download and use it.
- Founded in 2012 in Amsterdam by Shay Banon (CTO) and Steve Schuurman (CEO) and two others
- 2012 - $10m A-round funding
- 2013 - $24m B-round
- 2014 - $70m C-round
Shay Banon, Elasticsearch co-founder and CTO.
The background is Big Data and the need to be able to store, access and analyse multi-petabyte stores of information that come from a variety of structured and unstructured sources, such as ERP databases (classic structured data) through emails, Word documents, and on to social media postings on Twitter (classic unstructured data).
Legacy data warehouse/business intelligence tools can't cope with this multi-sourced, multi-formatted data because they are rigid, tending to rely on pre-defined data schemas for example, and formal ETL (Extract, Load, Transform) processes.
DDN's CTO Jean-Luc Chatelaine gives a good description of their limitations in a blog.
Elasticsearch sprang up to cope with this problem. To understand what it's doing we need to comprehend the open source big data entities in the IT landscape it inhabits.