Devilishly fast flash data access is on its way, courtesy of SMART Storage interfacing a DIMM-like flash product to a host's memory bus using Diablo Technologies' MCS TeraDIMM technology
SMART's ULLtraDIMM product has what it says is ultra low latency compared to existing SSDs and PCIe flash cards. It inherits all of the MCS TeraDIMM's low and consistent latency.
SMART Storage's ULLtraDIMM
SMART says its ULLtraDIMM, which comes in 200GB and 400GB capacities using 19nm NAND, has less than 5µs write latency, and can do 150,000 read IOPS and 65,000 write IOPS. The thing can run at ten full drive writes a day for five years and has a 2 million hour MTBF rating.
The IO performance scales as more ULLtraDIMMs are used but the latency stays constant. SMART claims the latency is half that of existing PCIe flash products. The latency is lower than a PCIe flash product because the I/O path to the memory bus is shorter with the MCS design and therefore quicker.
SMART's ULLtraDIMM's constant latency as IOPS rise
SMART Storage says its Guardian technology enables MLC NAND to be used instead of more expensive enterprise-grade MLC flash, enhancing its endurance to a 10 drive writes/day rating.
SMART says that two servers sharing a 1.2TB PCIe SSD using eMLC and SLC flash can get up to 225,000 IOPS from the flash, and can run 128 threads with 2,000µs latency. Fitted with four 400GB ULLtraDIMMs per CPU, the servers get 1.2 million IOPS from the flash – that is, four 150,000 IOPS per ULLtraDIMM – and can run 1,024 threads with 700µs latency.
SMART puts it like this: fit the servers with 2½ times more flash and get six times the performance (flash IOPS) and ten times the work done (threads and latency numbers).
We emailed Esther Spanjer, SMART Storage's technical marketing director, about ULLtraDIMM performance, pointing out to her: "You would expect the Diablo/SMART ULLtraDIMM to answer more data access requests than the P420m, meaning I/Os per second (IOPS). Yet this is not so. A 1.2TB P420m does 750,000 IOPS whereas 1.2TB of ULLtraDIM does 450,000. Why is that? I believe both products use the same sort of MLC flash. Why isn't the ULLtraDIMM faster overall than the P420m in IOPS terms?”
Her answer was:
The number of flash channels being utilized at any one time is one contributor. The controllers on the ULLtraDIMM utilize less flash channels as there are physically less flash placements on the DIMM module than there are on a PCIe card. Another factor is probably the architecture itself. The Micron PCIe card uses a native PCIe controller, which translates PCIe protocol/signals directly to flash chips. The ULLtraDIMM, being a first generation that uses the DDR interface, uses a protocol converter. We have two chips on the DIMM module: a DDR3-to-flash controller, and then a flash controller that talks to the flash chips.
Turning to cost/IOPS, Spanjer said:
Another way of looking at this is how much it would cost you to reach that IOPS with an ULLtraDIMM. Since the ULLtraDIMM is priced at a lower point that PCIe, you can actually increase the number of ULLtraDIMM’s to reach the desired IOPS (in this case, would need 5 ULLtraDIMM’s to reach 750K IOPS).
What then becomes very important is that we can increase the amount of ULLtraDIMM’s in a system and scale I/O, without impacting latency. The reason for this is that there is no bus contention on the DDR bus, something which is not possible with PCIe-based devices as they always have to go through the I/O hub and that will cause bus contention and increased latency. That’s the advantage of connecting directly to the memory bus, and no other device on the market today has that advantage.
… The Micron [P420m] PCIe card does not show any endurance specs, so I can only assume that they use commercial grade MLC flash without any endurance capabilities. Even if you assume parity on $/GB, we’re still substantially more cost-effective on a TCO basis due to our increased endurance. That means that on an ongoing basis, you could double the DIMMs to 2.4GB and still be paying less over time than for the Micron card.”
ULLtraDIMM drivers are available for Windows Server 2008/12, Centos, RedHat, and SUSE Linux, and VMware. There is no need for application software to be changed to use these ULLtraDIMMs.
Potential application scenarios are, as with the Diablo story, financial services, database and cloud, virtualisation, blade servers and big data analytics, either with ULLtraDIMMs used as block devices and/or as a memory extension.
SanDisk is buying SMART Storage and, since SMART is a user of Toshiba NAND, and Toshiba and SanDisk are partners in the NAND foundry business, SanDisk is buying one of the partnership's customers, in a sense.
SMART says ULLtraDIMM qualifications are underway with Tier-1 OEMS and Fortune 500 enterprises.
This product technology potentially gives SMART ULLtraDIMM flash products a significant data access speed advantage. Here is something that could possibly provide equivalent speed to software-enhanced Fusion-io flash without the added complexity of Fusion-io software. ®