Startup iguazio launches NVMe-propelled missile at enterprise analytics

Wall St-beating PaaS for Big Data firm touts crazy performance claims

iguazio’s Data-as-a-Service Enterprise Data Cloud converges different storage access protocols and use cases behind an access abstraction layer and claims to out-perform Amazon and all-flash filers at lower costs.

The startup’s pitch is that Big Data analytics data, input, extraction, loading and storing is grossly complex and inefficient with multiple steps and data copies exposed to users when they shouldn’t be, which delays data analytics processing and makes prices high and complicated to understand. Instead it proposes the use of a software platform that virtualises and abstracts away the complexity, with better storage software and media use.

Yaron Haviv, iguazio co-founder and CTO, tells us: "We took our long HPC, Networking, Storage, DB, Security, Management .. experience to build a stack from the ground up bypassing most of the legacy OS elements, producing the world fastest and lowest cost multi-model DB engine (Gartner and others' definition for DB that store and index different data types), with a stateless/pluggable API proxy layer above that emulate multiple existing APIs/Protocols (no need to modify apps.)"

The Enterprise Data Cloud (EDC) software provides a PaaS (Platform as a Service) facility, and the performance and cost is pretty far out. It claims that, with only four data appliances, companies can store up to 10PB/rack, with costs starting at $0.03/GB/month. The platform delivers 10 million transactions per second and throughput of 50GB/sec with sub-100 microsecond application latencies, across streaming, NoSQL, objects or files. NVMe flash is part of the reason for that.

EDC comprises a central software layer with:

• Four usage access methods
• • File and object - S3, Swift, NFS, HDFS, Linux FS
  • Streaming - Kinesis, Kafka, Spark DF
  • Key/value and records - DynamoDB, Time Series, Spark DF
  • Customer services - Docker and event-driven or scheduled tasks
• The core software has:
  • Fine-grain data firewall
  • Fast unified data processor for dynamic indexing, search, ACID, procedures, data optimisation and lifecycle
  • Virtual memory for metadata and cache - Uses NV Mem + NVMe flash clustered over 100Gbit RDMA
  • Key/value API access to direct-attach non-volatile memory, flash and disk plus remote disk and the cloud

The company says it offers the lowest cost/GB for such data with its consolidated data silos and dynamic tiering. The performance claims are dramatic:

• 60x S3 ops/sec versus AWS or others
• 100x stream put/sec versus AWS Kinesis per shard
• 25x NoSQL ops/sec versus Cassandra and DynamoDB
• 10x NAS IOPS versus all-flash arrays

A couple of charts add more substance to the performance claims;

Haviv said: "The fact we are a DB engine on bare-metal (no internal FS or block layers) means all the indexes/metadata/data are encoded/stored for optimal search with a data aware storage hierarchy. We do nice tricks like run fast SQL queries to look for files with specific metadata or content, no need for directory traversal. It also means we run various analytics tasks directly on our systems and offload critical App processing."

The cost claims are equally impressive:

• 15x lower NOSQL cost vs DynamoDB
• 15x lower cost for Stream vs AWS Kinesis
• 100x lower in-mem DB cost vs AWS ElasticCache
• 10x lower NAS cost versus all-flash NAS

Haviv told us: "Our extremely dense design (4 x 100GbE + 8-24 x NVMe per server + JBOD/Fs) drive very low-costs/GB, coupled with our unique Biz model it means 10x lower total cost/month than most storage vendors and AWS."

Users can, it claims, have a magnitude higher performance than AWS Redshift within a day of integration. Asaf Somekh, co-founder and CEO of iguazio, says: “Our beta customers have realized significant improvements in their data access speeds while also drastically reducing costs and simplifying operations.”

Its software can be deployed either on-premises or in hybrid cloud architectures, with self-service portals and APIs for developers and operators.

With iguazio’s EDC, stateless cloud-native apps provision data in seconds and consume it through standard APIs. Developers and business units share data, search and monitor and provision data lifecycles through abstract policies with multi-layered, role-based access control.

Equinix has a partnership with iguazio. An end-to-end real time demo is taking place at Strata+Hadoop World, booth 658, Sep 27-29. The iguazio EDC product is in early use at financial services companies, cloud-based services companies, and large-scale IoT deployments. It will GA in the second 2017 quarter. ®