Analysis Object storage has failed to to make it big time and is becoming just another way of storing files and a public cloud on-ramp.
Ten years or so ago the object storage startups said file storage, with its file:folder access scheme, could not cope with large volumes of unstructured data. It wasn't reliable enough and used up too much capacity with RAID data protection schemes, while high-capacity disk rebuilds took too long.
So along came Amplidata, Caringo, Cleversafe, EMC's Centera, Atmos and ECS, Object Matrix, Scality and others. Their technology used content addressing and, generally, erasure-coding, and offered near-limitless capacity, simple access with three hops or less, and less wasted capacity overhead.
But access was slower than with a filesystem and applications had to use object storage APIs.
Fast-forward to today and a new generation of scale-out file storage startups are challenging these assumptions about file system limitations, and building high-capacity and scalable filers. Think of Elastifile, Qumulo and Weka.IO, all working with standard file system access.
The file system resurgence has delivered a sustained and tremendous shock to the object storage market. You can count the object storage vendors who don't have a file access functionality on the digits of a fingerless mitt.
Now file system functionality and presentation is becoming more deeply embedded in object storage as well as it becoming an under-the-hood component of hybrid cloud data management.
These trends are shown perfectly by fresh developments from SwiftStack and Igneous.
Open source object storage software vendor SwiftStack recently announced its involvement in multi-cloud data management – we just love marketing messages.
The nitty gritty is v6.0 of its software with a single namespace covering on-premises and public cloud deployments and both file and object access methods without needing a file gateway.
The previously announced Cloud Sync pumps on-premises data out to the AWS and Google public clouds according to settable policies. SwiftStack says it uses cloud-native technology and cloud-native formats. Data can be read and written using NFS and SMB protocols, as well as object protocol and accessed on premises or in the cloud by any of these methods irrespective of how it was written.
We're told legacy applications with large unstructured data sets can now access and consume the same data from the single namespace via file services without the need for a gateway or application.
SwiftStack CEO Don Jaworski says object storage has: “grown beyond on-premises object storage to genuine multi-cloud data management.” It then has to compete with other multi-cloud data management. products whether they ate object storage ones or not.
Object storage startup Igneous is also moving into the hyperscale and hybrid file data space, initially using its hardware product as a filer backup target with more to come.
Having built a scalable object store using individually addressed Nanoserver disk drives, Igneous has developed a backup and archive application that uses it as a backup target with tiering and replication to the cloud from the on-premises Igneous box.
It is a tad unfortunate as the world is not short of backup systems that tier to the public cloud. Igneous is focusing exclusively on NAS data with a hyperscale aspect, potentially at the hundreds of billions of files level. It says it has no interest in protecting block storage.
Kiran Bhageshpur, co-founder and CEO, said that the on-premises system is a 4U storage node with two 1U servers, raw white box Intel servers offering stateless raw compute, layered on top. The software that runs in these servers is cloud-native, containerised, and uses Kubernetes. Supported filer sources include Dell Isilon, NetApp ONTAP, and Pure Storage's FlashBlade.
There is no need for backup media servers or array to array replication in Igneous's system. There is also just a single backup modality with archive data viewed as having a backup policy with retention set to forever.
Public cloud targets are AWS S3 and Azure Blob services, with Google Nearview coming, all with native object storage. Customers arrange their own cloud subscriptions.
Bhageshpur said the system kills the idea of backup windows. It can ingest data quickly because it has multiple write streams to the individual Nanoserver drives. He said: "I don't need low latency, I need throughput lanes," and multiple parallel write streams to the drives.
End users can search for and restore data themselves, and there is read-only NFS access to the backed-up data.
The technology looks very good but it seems as if the initial object storage product has simply became a base for building yet another on-premises backup target with a cloud back-end. It needs to be significantly better than existing products, and there are a lot of them, if it is going to be successful.
Object storage technology is becoming a component of filesystems and a data channel to secondary public cloud storage. On its own, raw object storage is not enough and can only succeed as part of the on-premises file environment and the public cloud. ®