Alluxio, the multi-silo Big Data access accelerating startup which did a deal with Huawei last year, has signed a similar one with Dell EMC for its ECS product.
The problem Alluxio fixes is that many separate storage schemes exist for large-scale unstructured data that needs to be accessed by many analytics-style apps. Each app (Hadoop, Spark, Storm, samza, etc.) needs to make its own arrangements to access each data source (AWS S3, HDFS, Ceph, Isilon, Gluster, etc.).
Alluxio provides a central software layer enabling any one of these apps to access any one of the data sources using their selection from a few interfaces. In a not too far-fetched way it does what NVMe drivers for for PCI protocol flash data stores in providing a standard interface for upper-layer system software.
As well as providing unified access Alluxio, open source code by the way, provides in-memory caching to speed data access, both read and write.
The Dell EMC deal with Alluxio involves the startup's Enterprise Edition (AEE) and Dell EMC’s Elastic Cloud Storage (ECS) object storage server for big data workloads. ECS was once code-named Nile. Dell EMC and Alluxio claim that their pairing “gives customers an alternative to DAS-based storage by providing the benefits of ECS together with the performance enhancements from Alluxio’s AEE software.”
Haoyuan Li, Alluxio's CEO, said his firm enables ECS to be used as a gateway or as a a storage component: “Alluxio gives Dell EMC ECS customers total flexibility to use any compute framework to access disparate storage systems on premise and in the cloud at memory speeds.”
For potential customers it might be worth comparing the Dell EMC - Alluxio combo with Dell EMC’s DSSD D5, which also provides fast access to Big Data. A speed and cost comparison between the two would be interesting.
For Alluxio the signing of Dell EMC just a few months after signing Huawei is good news, and other storage suppliers will likely follow suit, like HDS, HPE and NetApp for example. You can try out Alluxio SW here. ®