This article is more than 1 year old
Will open source storage make the hyperscale dream real?
One small Ceph, or one giant leap?
OpenStack and me
Key features include the option to run Ceph's file journals on solid-state storage for better performance, and the ability to do controlled rolling upgrades. “For example, we upgrade the controllers first, then the back-end, then the OSDs (object storage devices),” Gerasimatos says. “We just set a flag on the node, it dumps its data to the others, we do the upgrade, then move on to the next. It's not as dangerous as a full upgrade.
“Ceph is the darling of OpenStack – more people use it there than any other storage layer,” he continues, adding that the one blocker to Ceph adoption he sees among his finance industry peers is when they still have undepreciated legacy storage.
In theory this should not be a problem, because one advantage of software-defined storage is that it is hardware-independent, but the business reality is rather different. “Bringing legacy hardware into Ceph is possible,” he says, “but it's very difficult to move running application instances.”
Both the all-flash arrays and Ceph do have their strengths and weaknesses, Gerasimatos adds. For example, even with SolidFire's built-in compression and de-duplication, you don't want big datasets living on flash. Conversely, Ceph is less well suited to transaction-heavy database works, such as an Oracle application doing 100,000 inserts and 100,000 row-deletes, or cases where you have bandwidth-greedy 'noisy neighbour' applications.
While FICO adopted Ceph via Red Hat as part of a bigger Linux and OpenStack project, DARZ took an appliance approach, buying Ceph-based Fujitsu Eternus CD10000 storage systems. Based in what were once the central bank's gold vaults in the German state of Hesse, DARZ (Darmstädter Rechenzentrum, or Darmstadt data centre) is a cloud service provider selling storage and other OpenStack-based services on the provider-agnostic Deutsche Börse Cloud Exchange (DBCE) marketplace.
As DARZ's head of sales & IT operations Lars Göbel explains, it was the company's system integrator Profi AG that proposed the hyperscale appliance approach. Not only is the Fujitsu hardware well supported locally, but the Ceph software allows it to scale to multiple petabytes at relatively low cost, with each new node adding both capacity and performance linearly, and it can provide storage in block, file or object format.
“This is the complete commodisation of IT resources. If this model is successful then the age of high storage costs will be gone,” claims Frank Reichart, the senior director product marketing for storage solutions at Fujitsu Technology Solutions.
Göbel is particularly pleased with the CD10000 and Ceph's multi-client capabilities, which ensure customer separation. “This secure and powerful infrastructure means that we can join the DBCE marketplace and open up an innovative sales channel for our cloud services,” he says.
With so many Ceph implementations being part of OpenStack private cloud projects, it's no surprise that Red Hat's Sage Weil sees this as a major opportunity. “We have a lot of traction in the OpenStack space so it's an important project for us,” he says. “About two-thirds of all OpenStack deployments use Ceph RBD, according to the latest user survey. That's more than twice the next option, which is local storage with LVM.” RBD, or RADOS block device, allows Ceph to be mounted as thinly-provisioned block storage to support virtual machines.
Those tricky teenage years
The software continues to evolve, too. It recently received its eighth major release, codenamed Hammer, and more is planned for the next, including better support for long-distance replication. Nick Gerasimatos says he is particularly looking forward to the latter – at the moment each of FICO's regions replicates locally between active-active Ceph pairs in nearby data centres, but geo-replication is more of a challenge.
“Ceph is a bit like a teenager – it's still growing,” he jokes, adding that he would also like to see data de-duplication and encryption for data in flight, and a faster, more robust underlying file system – he points to Red Hat's other storage platform, Gluster, as an example of the latter. “I would love to see a merger of Gluster and Ceph, but that's not going to happen,” he adds.
Still, the whole open-source storage area is continuing to grow, both in size and sophistication – even more so if you add in the free object software elements and what Eric Slack calls “OSP, the open storage platform,” which is hyperscale and is more commodity than closed, but is not necessarily open-source.
For instance, even disk drive developers are getting connected. Seagate earlier this year open-sourced its Kinetic platform for developing Ethernet-connected hard drives that talk objects rather than blocks. The Open Kinetic project is now under the auspices of the Linux Foundation, and both Seagate and Toshiba have demo'd drives. While not a Kinetic partner, HGST, now part of Western Digital, has also shown directly-addressed object storage drive technology.
The idea with all of these is to reduce storage costs by removing the need for SANs and arrays, and instead enabling the likes of Ceph, Gluster and Swift to talk directly to drives that are native key/value object stores. In effect each object-addressed “drive” becomes a miniature OSD storage node, including multiple storage elements – Toshiba's are hybrid flash/disk units, for example – and a supervisory processor.
Of course this won't remove the need for the likes of Ceph. Something still needs to track which objects are on which drives, and provide file and block access for those applications that need it. And of course open-source software is never entirely free – and neither for that matter is free software (the free here refers to the lack of restrictions on usage and distribution, not the price). You just have to decide how much you want to pay for it, and who to, even if the cost is merely staff time to seek community support.
Still, there is no doubt that open-source, allied with the software-defined movement, is dramatically changing the face of storage. From here on, nothing will be quite the same. ®