This article is more than 1 year old

One bit to rule them all? Forget it – old storage types never die

Pups' guide to running with the storage pack

Except that it's all different today

Of course, all of that might or might not be completely irrelevant. Most object storage is complicated software presenting storage to developers that isn't quite as complicated as file systems but is designed to be rather heavily redundant and distributed, and is ultimately just a different layer between the developer and the raw block storage of the drive than a file system is.

But object storage isn't necessarily distributed. Things like Seagate Kinetic drives exist. Kinetic drives don't present an underlying raw block storage to anything.

This means that if developers want redundancy they'll have to either code that into their application themselves, or they'll have to use an object storage system that understands Kinetic drives as the fundamental base storage device instead of blocks. As discussed above, that could impact how replication/distribution of data is handled.

Block storage has been used by a number of companies that have managed to build highly resistant distributed systems. Some have even been successful, building file replication systems that can handle any type of file, though admittedly not many.

Where to use what

Block storage is not well suited to large-scale set-ups. An individual block storage device isn't all that big. You can create a virtual block storage device (VHD or LUN) that spans multiple physical block storage devices, but it's still one "thing".

If you are doing exact block-for-block replication of identical physical devices (RAID 1 or Redundant Array of Independent Nodes (RAIN) then replicating blocks is really straight forward, but your block device will only ever be the size of the physical devices in question. In any other situation there is an index of what logical blocks are mapped to what physical ones.

Here, size is a problem. The larger your block device, the bigger that index. Therefore, the more likely it is that corruption of that index can be a problem, and the longer a full rebuild of your block device will take in the event of a disk or node failure.

Perhaps more to the point, today's servers and operating systems only understand block devices that are so large. It may be technically possible to build a distributed system with a single virtual block storage device that is multiple petabytes in size, but what can talk to it? And how well does it handle failure?

So we break our block devices down to more manageable sizes. By all means make a multi-petabyte distributed storage system, but hand out the block devices a few terabytes wide, at best.

This makes block storage ideal for virtualisation. Each VM, or group of VMs can have a LUN of their very own. That can be distributed among the storage cluster and Bob's your uncle. Of course, managing block storage for tens of thousands of VMs can get very tedious, very quickly.

Object storage, by contrast, is absolutely terrible at this. The whole point of it is to make distributing billions of relatively small chunks of data across a storage array of unlimited size relatively simple and straightforward. It's not natively good at replicating large files, and of the shims that exist today, very few are good at turning any object storage into something you'd want to run production tier-1 VMs on.

That leaves us with file systems. File systems are far easier to use than both object stores and block storage. File systems are designed to handle multiple files and are also designed to allow the contents to change size. File systems also allow for lots of increased features around security of the individual files, with the downside being that they're largely useless for storing more than 10m files. In theory, even NTFS can hold more than 10m files. In practice, even ZFS turns to glue when you actually try it.

So file systems are great for holding a few million small files, tops, or for holding VMs where there is either going to be a high turnover in the number of VMs or VM size is likely to change with some frequency. In the file system world there is none of this "assigning a LUN to every VM" nonsense, nor wasting time resizing LUNs.

Of course, replicating file systems isn't easy. That means generally file systems that need to be distributed and highly available will exist on top of a distributed block storage set-up.

While it's possible to put a file system on top of a distributed object storage setup, in practice, that's pointless. The object storage system is going to be much better than the file system at storing small files and kind-of-crappy to unworkably useless at running VMs off of a filesystem running on an object storage shim.

The above is why no one storage type has won out over all the others. Each has its place. ®

More about

TIP US OFF

Send us news


Other stories you might like