As virtualisation brings IT consolidation back onto the agenda, storage federation is being touted by many as the best way to impose discipline on infrastructures that have grown unwieldy.
So what can it offer that storage virtualisation or a global file system cannot, and is it time for us all to join the federation?
Storage federation aggregates multiple storage systems, typically by adding distributed volume management plus peer-based control and locking mechanisms so they can be treated as one, while still letting them operate as discrete peers within the infrastructure.
Proponents say it allows you to move applications between systems (via native peer-to-peer communication), or add and remove storage systems over time, offering scalability, availability and ease of management.
It means you have the right data in the right place, and that it is protected properly.
“There are two reasons for turning to federated storage,” says Craig Nunes, formerly of 3PAR and now storage marketing vice president at HP, which bought 3PAR in 2010.
“One is asset management. I may have three storage systems, two are growing very fast, one isn't and has a load of available capacity. The complexity of migration has prevented people simply dragging and dropping workloads to the spare capacity, and it ends up being easier just to upgrade the other two.
“The other is quality of service management, especially in an age when things are shifting to flash. Workloads are evolving to the point where they could benefit from the flash on your new array if you can move them, while older applications might not need flash so much and could be moved off.
“People also like federation for technology refreshes, to avoid the headaches of migration.”
Nunes suggests that one thing distinguishing storage virtualisation from federated storage is that federation is peer-based.
In contrast, virtualisation technology such as EMC's VPLEX, Hitachi's USP or IBM's SVC uses an in-band gateway or appliance to aggregate and manage multiple back-end arrays, making them look like a single storage resource.
“Storage virtualisation is good for management and stuff. There are benefits there,” he acknowledges. “The problem is you may be unable to comprehend the characteristics of the underlying LUN or hardware – what we call a blind LUN.”
Another increasingly useful option is to bring other forms of storage into the federation, most obviously hybrid cloud storage.
Indeed, the hybrid cloud is already a form of federated storage. In a truly federated environment, data would be able to move independently of the platform, the hardware or the software.
This also means that data could move into and out of a cloud provider, all without an end-user or application having any knowledge of what is happening.
Noam Shendar, chief operating officer at Zadara, which uses a cloud-federated approach to deliver enterprise on-premise storage as a service, argues that this is a problem for much of the storage industry.
“Most companies have taken sides – either the traditional incumbent model or the cloud – and are forcing the customer either to choose one of them or operate two models,” he says.
He adds that this is because the two revenue models are so different that they are hard to combine: one is a straightforward sale, the other is a continuing rental-type income.
“The incumbents are afraid of this for two reasons. One is revenue shifting into the future and the other is the pressure on gross margins,” he says. “Shifting revenue in the future is a better model overall but it needs patience, and US publicly-traded companies don't have patience.”
Of course, some companies have tried to cross this chasm. NetApp offers Cloud ONTAP, which can link NetApp hardware filers with a virtual filer in the Amazon cloud, and Microsoft’s StorSimple gateways seamlessly bridge on-premise storage to the Azure cloud.
“Federated today is cloud, on-premise, off-premise and so on,” says Shendar. “I need the ability to move or replicate my data when I need it. For example, the data is in Ireland and you want it in Germany, you press a button and it moves. Press another button and the old copy is deleted.”
Recalling data from the cloud can be a problem, depending as it does on the available bandwidth
Of course, recalling data from the cloud can be a problem, depending as it does on the available bandwidth, and ingesting large amounts of data is also potentially problematic.
A good storage federation story could therefore include physical arrays, cloud storage, a virtualisation layer for storage pooling, software-defined storage and a way of accelerating data to and from the cloud.
So, if the cloud storage gateway and the hybrid cloud are already both aspects of storage federation, and if spreading that goodness across the enterprise offers so many advantages, federated storage is the future, right? Well, not quite.
There are other technologies vying to solve many of the same problems, most notably storage virtualisation and global file systems. And indeed, there is considerable overlap between them, as Shendar notes.
“Some level of storage virtualisation is a necessary underpinning [for federated storage], although you could in theory federate physical storage, as NetApp Private Storage does,” he says.