Sysadmin blog Ready to hear about the next IT startup niche? It's an emerging market called "host-based caching", and you might wish to pay attention if you'd like to understand how a few people got stupidly rich a few years from now.
The Cliff's notes version of host-based caching is that it turns flash drives inside your virtual hosts into cache in order to make the SAN seem less slow. This stuff is interesting to an ops guy like me, so here is my 10km-high view of the "host-based caching" software space.
We'll start the list with Proximal Data, which has a host-based caching offering it calls Autocache. VMware has integrated a host-based caching into ESXi 5.5 with vFRC. SanDisk's FlashSoft is a player as well, again with its own particular twist on things.
The loudest of the bunch is PernixData. PernixData pride itself on doing host-based write cache while the others currently offer host-based read cache.
I have had a chance to play with all of these offerings thus far and have found each one of them are good products. They make a real-world difference to small- and mid-sized networks. Their impact on larger networks is still largely an unknown, though Proximal Data have had some sales wins recently that are providing evidence showing host-based caching works well at scale.
The different offerings in this space have their own strengths and weaknesses; the market is young enough that only one of them – VMware – is making a play based on enterprise support and "integration". There is still so much innovation to do that the rest are competing on features, applicability and so forth. We simply aren't at the point yet where "one vendor is more or less like another."
PernixData is dwarfed in size by both FlashSoft and VMware. In turn, PernixData's resources outstrip those of Proximal Data by an order of magnitude. I'll examine in a future article exactly how these size and resource disparities make a practicable difference.
If you want to do it properly, host-based write caching creates a new tier of storage in the data centre. I remember saying as much to PernixData's CTO Satyam Vaghani during a beta test of its product. I thought PernixData's offering was amazing then, and I still do.
I remember very specifically writing an email that said if people ever realise what PernixData is capable of, we'll be redesigning our data centres to take advantage of it. Satyam evidently agrees, because he talks about this enthusiastically at VMUGs and in his own blogs.
What might seem off to most is that Proximal Data (which does not currently offer a write cache with its product) is in violent agreement with all of this. The difference of opinion between the two companies - which is the basis of the differences in their products - occurs in whether or not a new tier of storage is desirable.
The Proximal Data argument is that with a read cache you simply add flash, enable the caching software and things go faster. No fuss, no muss. By caching hot reads at the host you are taking load off the SAN. This makes the SAN more responsive, not just for the host with the cache, but for all hosts attached to that SAN.
It took a while for me to come around to this train of thought. I really liked PernixData and write cache just seems so much cooler than read cache. It certainly looked nice when I used IOmeter for benchmarking. Large sexy number nerd porn!
A niche by any other name
It took a very long conversation with Proximal Data's CEO to finally convince me Proximal Data was in the right. What works in my testlab, he said, has to be considered at scale. A new tier of storage has risks, imposes operational changes, can end up requiring its own NICs for replication traffic, and ultimately can get to the point that it needs its own priestly caste of nerds to tend to it.
PernixData argues the benefits outweigh the risks and the costs. Proximal Data argues that the risks and costs are such that read-caching is the mass market play. This can devolve into a rather silly game of "you're a niche, no you're a niche!" Toss in some very strong personalities, big egos, high stakes and Twitter and you have a startup-dominated IT niche not so very different from any other.
Personally, I fall somewhere in the middle. I think that write-caching is a significant enough niche to get "crazy stupid rolling around in it" rich, much like Apple Macs are "just a niche." Meanwhile, read caching holds the mass market appeal and is also a means to getting "crazy stupid rolling around in it" rich.
Proximal Data says it is working on write-caching and will add it "when they feel [they] have found a way to make host-based write back caching as risk free and non-disruptive to data centre operations as is possible."
PernixData allows setting either read- or write-caching on a per-VM basis. VMware has been somewhat less direct about timeframes for feature expansion, while SanDisk's FlashSoft are promising write caching "soon".
As I mentioned before, VMware's vFRC play is integration. A huge part of that is "it comes with enterprise class support", as well it should; it's only available as part of the Enterprise (or higher) versions of ESXI 5.5.
VMware's making the right play. Everyone else on the market is easier to use, but lots of enterprise buyers will get a whole lot more warm fuzzies from the VMware name than they will get from making the lives of the nerds easier.
Such is the way of things. Still, VMware sells an awful lot of Enterprise and Enterprise Plus licences, making this "free" for those users. It's a built-in customer base that can't be ignored, but it is self-limiting. vFRC alone will not be selling many new Enterprise Plus licenses.
PernixData has that write cache, and for certain workloads it absolutely sings. At $7,500 per host just for the software (we'll leave Proximal Data's arguments about the costs of a new tier of storage to one side) I start asking pointed questions about how expensive your storage must be if you are willing to pay $7,500 per host to make it faster. PernixData gets a "Thank You" from me, however, for taking SMBs into consideration; it offers a limited SMB package of up to four servers for $9,999.
Proximal Data has been laser-focused on read cache for two years. It does only the one thing, but it does it very, very well. It is the cheaper option (remember, mass market play) starting in at $999 per server for caches 500GB or lower. It gets bonus points from me for committing to supporting Hyper-V by the end of the year. Additional bonus points for building in robust role-based administration aimed at making like easier for cloud service providers.
Philosophically, I also buy into the Proximal Data philosophy that caching should be "fire and forget": where per-VM configuration is an option, not a requirement. I have enough stuff to manage, I don't want to add another tier of storage or another layer of management software on top of that. More pips to them there.
SanDisk's Flashsoft is newer to the host-based cache market, but old hand at the server caching market. Flashsoft comes in at $3,000 for Windows, Linux for Linux and FlashSoft $3,900 for VMware. It is busy beavering away on write-caching for VMware and is fond of things like write combining.
SanDisk gets a nod from me for a company-wide commitment to cache – while not part of the same team ReadyCache is a product I have used to great effect. SanDisk would not have bought FlashSoft if it didn't feel it would enable people to make better use of SSDs, thus allowing SanDisk to sell more SSDs.
Where are your blocks now?
What you store your data on makes a difference. VMware supports acceleration of both SAN and NAS protocols. PernixData supports only SANs. Both Proximal Data and Flashsoft can accelerate DAS, SAN or NAS datastores.
A lot of companies have had servers – especially certain big-name blade systems – sold to them with SSDs inside they can't really use. What good does that SSD do someone using VMware on a SAN? Use VMware's swap to SSD feature? Please. Talk about niche applicability.
There are millions of these things out there right now in data centres not being used. Any of these technologies could be used in a "install software, enable, go faster" write-cache mode. (Well, VMware is a little more complicated in its implementation…)
For smaller shops, putting an SSD into an existing server is entirely doable. It's a quick and easy way to extend the useful life of that server. If you do it to enough servers you can even extend the usable life of the storage you're using to feed those servers. The makes host-based caching an evolutionary approach to data centre performance issues.
On the other end of the spectrum are converged players like Nutanix. It can meet your needs for performance, but to get there it fuses compute and storage into a single unit and scale both out together. That's radical thinking for an industry that long ago divorced compute from storage. (Though to be fair, it's doing rather well.)
With so many options, it's a great time to be a storage admin. No matter which company emerges victorious in the flash wars of the 2010s, there's enough healthy competition that it looks like end users will be the winners. ®
 One of the companies discussed in the article - Proximal Data - is a client of my company, eGeek. We help them with content marketing, community outreach and social media. That aside, I'm writing about this stuff because it's cool; Proximal Data was not offered the opportunity to vet this article before submitting.
 Write combining gives me the heebie jeebies. Take the link with a grain of salt - it is a blog about the topic on a competitor's website - but do read it. I promise you that rather a lot of research went into it... and there are "I did this in my testlab and it blew up" reasons for my heebie jeebies.
FlashSoft has told me that regardless of lab results, it simply isn't an issue it encounters in real world deployments and I shouldn't be worried about it. There's some logic to that too; it's worth further research. ®