Kaminario 'shelves' future. Between you and NVMe, it could get flashy
Fabricating array structure for more compute, storage separation
Analysis All-flash array vendor Kaminario has been in business a year longer than Pure Storage but is some way behind in sales, size and funding. However, it has begun expressing a technology vision that could accelerate its growth dramatically.
We don’t know what Kaminario’s revenues from its K2 scale-up and scale-out arrays built from k-block modules are, withIDC telling us: “Kaminario has much larger revenues than Violin but we are not releasing those publicly.” [It is still a privately held vendor that does not wish to share that data.] Violin is the laggard vendor in the AFA market and its latest revenues would seem to imply a $40m/year product + services revenue run rate.
IDC includes Kaminario revenues in the “Others” category, giving that category $368.4m revenues in the first 2016 quarter. The lowest identified vendor, Oracle, had $3.4m, then there was Fujitsu at $7.8m and Huawei at $16m, with NetApp at $328.4m. and Pure at $341.1 million. Let's say Kaminario has more than $10m revenues (Violin) and probably less than $50m.
There is a renaissance of tech innovation in non-volatile storage and this provides an opportunity for the company to present itself as having a distinctive technology vision that will substantially improve its customers’ AFA experience.
Kaminario CTO Shachar Fienblit identifies five non-volatile memory trends driving the future of storage:
- 3D NAND
- Non-volatile (storage) memory - 3D XPoint, ReRAM
- NVMe Fabric
- NVMe network shelves
He also makes a few assumptions about how changes will affect things.
3D NAND (and possibly QLC) remain dominant. NAND density (cost) will continue. Adding more layers at same process geometry will greatly increase error checking and correcting capability means endurance will decline. Fienblit asserts that NAND will remain the leading storage media and will be the commodity curve that replaces HDDs in the data centre.
3D XPoint, ReRAM and possibly other similar tech will improve NVM performance, but NVM price will be much closer to DRAM than Flash. Also 3d XPoint DIMM ~1 microsec performance is much faster than flash but still far from DRAM. So these new NVM technologies are too expensive to replace NAND and should be used wisely. They are not going to replace NAND for 5+ years.
NVMe is much more efficient than SCSI and will soon replace SATA as the direct-attach method. The implication is a lower benefit for AFAs since these systems already aggregate the performance of several SSDs. The controller is the bottleneck so decreasing latency from disk to controller is not that interesting.
NVMe Fabrics extend the efficiency of local NVMe interface over the fabric. He thinks ROCE RDMA over 100GbitE will win out over the alternatives, especially RDMA over Fibre Channel. But customer adoption will be relatively slow.
He says he sees NVMe shelves coming. These are shelves (enclosures) with NVMe SSDs and RDMA network connectivity. This provides an interesting opportunity to further improve the decoupling of capacity from compute.
Kaminario's NVMe over fabrics shelves scheme
NVMe won’t bring a performance revolution in the near term, as the latency gains from direct SATA SSD access to NVMe SSD access (NAND MLC PCIe x4 Gen 3 ONR3) are only a lessening from c110 microsecs to about 95 microsecs, using Intel 2015 numbers.
A future NVMe PCIe x4 Gen 3 will have 20 microsecs latency. Fienblit says NVMe SSDs can deliver a 3-4x performance improvement if we eliminate data services and data reduction. NVMe Fabrics can v-cut external array access latency. Flash media remains a latency bottleneck until/unless it gets replaced by 3D XPoint.
New high-performance technologies are too expensive for mainstream use and will enter the market as niche, tier 0 applications. There won’t be a disruption of the core AFA market.
What Kaminario intends to do is:
- Use advances in NAND and continue to utilise the most cost-efficient SSDs while leveraging its own endurance optimisation capabilities
- Use 3D XPoint to replace DRAM for caching MetaData and Data for achieving better cost/performance
- Deploy NVMe fabric for external connectivity
- Use advances in NVMe, NVMeF, NVMe Network Shelves and make, he says, “a disruptive advancement in AFA scalability, cost efficiency and agility”
Fienblit envisages an AFA with an internal 100Gbit Ethernet backplane or fabric, connecting multiple compute (controller) nodes to multiple storage (capacity) nodes for a full decoupling of capacity and compute. Both can be scaled independently without effective limits. There can be globally shared spares of storage and compute. Performance and capacity are balanced under any expansion path without data movement.
Perhaps some of the compute resource can be exposed externally and used to run applications needing closeness to the storage,
This is the clearest and most detailed path to NVMe over Fabrics and XPoint use we have seen from any vendor. It could change Kaminario’s fortunes and bring it to much greater prominence as an AFA vendor. ®