All-flash hope-dash: The future is hybrid

But it will be an all-flash hybrid array

Comment In the future we will have an all-flash hybrid concept. This can’t be considered a prediction; it’s just a fact. If you look around, a large number of vendors are craving Intel’s 3D Xpoint memory and this will be the next tier 0 (or cache) for many newly designed storage arrays.

3D XPoint in short

3D XPoint is a new class of memory which sits in the middle between RAM and flash. It’s persistent (like flash); fast (not like RAM, but much faster than flash); dense (again, not like flash, but more than RAM); and durable (more than flash, less than RAM). And, of course, the price will be higher than today’s flash chips but lower than RAM. An interesting video about 3D XPoint memory was recorded at SFD8.

This makes the new non-volatile memory a good fit for any sort of landing zone for data in the storage array, where it can be easily reorganised, compressed, deduplicated and whatever else before being flushed on cheaper flash memory.

And I wouldn’t be surprised to see a new class of products taking advantage of this media. For example, think about what a company like Diablo Technologies could do by applying its technology to this kind of chip. (If you’re not familiar with Diablo yet, tune in for their session at Tech Field Day 10 next week.)

3D XPOint memory concept

3D Xpoint advantages in practice

I’m risking some oversimplification here, but 3D XPoint is not just about better IOPS and latency… it will have three major impacts in storage arrays.

• Higher density: It will be possible to implement a very dense back-end. Much more focused on durability and reliability than today. It doesn’t mean that today’s AFAs are not reliable, but tomorrow they’ll be able to use relatively slower writing speeds, less I/O channels and, consequently, denser chips and flash modules to obtain similar results. And it probably won’t impact reads, or not that much at least, and reading speeds will remain in the same range of current flash modules.
• $/GB: this is just a consequence of the previous point. Using denser, and possibly cheaper, flash will drive down prices. Today it’s TLC (3 bits/cell), but, sooner or later, someone will be producing QLC (4 bits/cell) and all the efficiency introduced by a 3D XPoint layer at the front-end will be fundamental to save write cycles and performance in the back-end.
• Fast NVMe over fabrics adoption: It’s useless having a great latency, unless it can be taken advantage of.

Closing the circle

Many vendors are already using NV-RAM in their arrays, but usually it is a component made out of standard DIMMs and capacitors. It’s very expensive and small in terms of capacity (because of the components needed to make RAM persistent in case of a power outage and the size of RAM chips themselves).

3D XPoint will be able to bring a much larger capacity to the non-volatile memory layer of these systems, enabling bigger caches and efficient tiering mechanisms.

Looking at how modern hybrid systems are designed, I’m wondering if some vendors, like Nimble or Tegile, will be in a better position than AFA vendors when this new media is available. Next February I’ll be meeting with both Pure Storage and Nimble storage and questions about how they think they’ll be implementing 3D XPoint in their systems will be on the top of my list.

At the end of the day, flash memory is just a fast medium, not a synonym for primary storage… and tomorrow, your next “hybrid” array will be better than your current AFA.

If you want to know more about this topic, I’ll be presenting at the next TECHunplugged conference in Austin on 2/2/16. It's a one-day event focused on cloud computing and IT infrastructure.