Analysis File lifecycle management – aka Hierarchical Storage Management – can work with data protection and secondary storage convergers, but only by getting the heck out of the data path... which is where Primary Data failed.
Curse of Woz strikes again – first Fusion-io fizzles out, now Primary Data goes downREAD MORE
That's the contention of Komprise, which has claimed that a widely distributed set of files on different storage systems can be unified into a virtual data lake and have their life cycles managed by its software.
Primary's metadata-driven abstraction layer, if you remember, had a bit of a TITSUP issue – by which we mean a Total Inability To Sell Users Products.
Komprise has said that its gear will allow customers to concentrate data protection efforts on files that need it while the unchanging stuff gets sent to cold storage by Komprise.
It claimed to provide a corporate-wide, indexed virtual repository of file data in both on-premises and public cloud silos, with its control plane off to the side of the data and metadata path.
Co-founder, president and COO Krishna Subramanian told us that unlike previous HSM player Primary Data, Komprise tech only acts on unstructured or file data.
The company distinguishes itself from other (unstructured) data management companies by its operation at the file system level, where it provides efficient file placement on various storage tiers that is transparent to applications. Other data management companies can then operate upon this file system layer.
Subramanian said customers could, for example, use Rubrik to back up all their secondary file data and then use that data for analysis, etc. But if Komprise is involved, the non-changing secondary file data can be archived off. It doesn't need backing up and the Rubrik – or Cohesity or Commvault or Veeam or some other data protector – doesn't waste resources by backing up non-changed data.
Komprise sees itself, logically, almost as part of the file system layer.
Architecting IT consultant Chris Evans said: "ILM (Information Lifecycle Management) and especially NAS has been around for years. The piece that always has me intrigued is how to move data without affecting the UNC (Uniform Naming Convention) of the original file.
"It's great if the UNC name is abstracted already or behind something like DFS [Microsoft's Distributed File System], but many users hard-code IP addresses and filer server names into content that makes it impossible to move without editing those source entries."
How does Komprise's technology move files transparently to applications?
Subramamian said that when Komprise moves a file it replaces the file with a standard file system construct which it calls a "symbolic link". A symbolic link is just a pointer (kind of like an URL) to the new location of the file.
She explained how it works at a basic high level when an application tries to open a file moved by Komprise.
The application goes to open a file that has been moved by Komprise and replaced with a symbolic link. The application opens the file exactly as it would open any other file, and the file system (SMB or NFS) handles the request. The application sees nothing out of the ordinary.
Behind the scenes, the application's request causes the file system to "follow the link" and forward the request to Komprise, which then responds with the content. This handshake is completely invisible to the application, since it is handled by the standard file system.
Subramamian told The Register symbolic links have been natively supported by NFS from its inception and by SMB since roughly 2008 (it has been supported starting with Windows Vista and XP) and so this approach today works for both Linux and Windows. In the past, it was not possible across both file systems and protocols.
The company has several patents pending on its tech.
Komprise, which has a customer list numbering over 100 that includes the Electronic Arts gaming company, has been growing quickly since it started shipping product about 20 months ago. It told us revenues were up 450 per cent in its first quarter compared to the same quarter a year ago – although of course we have no insight into the base number.
The CEO said it is adding capabilities to automatically move data from public cloud object storage to hot storage layers like Amazon's EFS, and to do this for AWS, Azure and Google. It can currently move data to tape using Spectrum Scale for IBM tape and Black Pearl for SpectraLogic tape. The LTFS tape file system is not supported by Komprise.
Where might future Komprise competition come from? Conceivably Cohesity, Rubrik and other data management companies could extend their metadata gathering and processing to provide similar capabilities to Komprise.
Until then this niche startup appears set to continue growing and will continually face a boundary definition problem: where does file lifecycle management stop and secondary data management start? If it steps across into secondary data management then it will start competing with the Actifios, Cohesities, Rubriks, etc. Careful strategy will be needed to avoid that. ®