For pity's sake, man, get out of the data path: Avoiding the Primary Data problem

Things that make you go hmmm... HSM firm talks up unusual approach

Analysis File lifecycle management – aka Hierarchical Storage Management – can work with data protection and secondary storage convergers, but only by getting the heck out of the data path... which is where Primary Data failed.


Curse of Woz strikes again – first Fusion-io fizzles out, now Primary Data goes down


That's the contention of Komprise, which has claimed that a widely distributed set of files on different storage systems can be unified into a virtual data lake and have their life cycles managed by its software.

Primary's metadata-driven abstraction layer, if you remember, had a bit of a TITSUP issue – by which we mean a Total Inability To Sell Users Products.

Komprise has said that its gear will allow customers to concentrate data protection efforts on files that need it while the unchanging stuff gets sent to cold storage by Komprise.

It claimed to provide a corporate-wide, indexed virtual repository of file data in both on-premises and public cloud silos, with its control plane off to the side of the data and metadata path.

Primary Data

Co-founder, president and COO Krishna Subramanian told us that unlike previous HSM player Primary Data, Komprise tech only acts on unstructured or file data.

The company distinguishes itself from other (unstructured) data management companies by its operation at the file system level, where it provides efficient file placement on various storage tiers that is transparent to applications. Other data management companies can then operate upon this file system layer.

Subramanian said customers could, for example, use Rubrik to back up all their secondary file data and then use that data for analysis, etc. But if Komprise is involved, the non-changing secondary file data can be archived off. It doesn't need backing up and the Rubrik – or Cohesity or Commvault or Veeam or some other data protector – doesn't waste resources by backing up non-changed data.

Komprise sees itself, logically, almost as part of the file system layer.

Architecting IT consultant Chris Evans said: "ILM (Information Lifecycle Management) and especially NAS has been around for years. The piece that always has me intrigued is how to move data without affecting the UNC (Uniform Naming Convention) of the original file.

"It's great if the UNC name is abstracted already or behind something like DFS [Microsoft's Distributed File System], but many users hard-code IP addresses and filer server names into content that makes it impossible to move without editing those source entries."

The tech

How does Komprise's technology move files transparently to applications?

Subramamian said that when Komprise moves a file it replaces the file with a standard file system construct which it calls a "symbolic link".  A symbolic link is just a pointer (kind of like an URL) to the new location of the file.

She explained how it works at a basic high level when an application tries to open a file moved by Komprise.

The application goes to open a file that has been moved by Komprise and replaced with a symbolic link. The application opens the file exactly as it would open any other file, and the file system (SMB or NFS) handles the request. The application sees nothing out of the ordinary.

Behind the scenes, the application's request causes the file system to "follow the link" and forward the request to Komprise, which then responds with the content. This handshake is completely invisible to the application, since it is handled by the standard file system.

Subramamian told The Register symbolic links have been natively supported by NFS from its inception and by SMB since roughly 2008 (it has been supported starting with Windows Vista and XP) and so this approach today works for both Linux and Windows. In the past, it was not possible across both file systems and protocols.


The company has several patents pending on its tech.

Komprise, which has a customer list numbering over 100 that includes the Electronic Arts gaming company, has been growing quickly since it started shipping product about 20 months ago. It told us revenues were up 450 per cent in its first quarter compared to the same quarter a year ago – although of course we have no insight into the base number.

The CEO said it is adding capabilities to automatically move data from public cloud object storage to hot storage layers like Amazon's EFS, and to do this for AWS, Azure and Google. It can currently move data to tape using Spectrum Scale for IBM tape and Black Pearl for SpectraLogic tape. The LTFS tape file system is not supported by Komprise.

Any competition?

Where might future Komprise competition come from? Conceivably Cohesity, Rubrik and other data management companies could extend their metadata gathering and processing to provide similar capabilities to Komprise.

Until then this niche startup appears set to continue growing and will continually face a boundary definition problem: where does file lifecycle management stop and secondary data management start? If it steps across into secondary data management then it will start competing with the Actifios, Cohesities, Rubriks, etc. Careful strategy will be needed to avoid that. ®

Other stories you might like

  • Deepfake attacks can easily trick live facial recognition systems online
    Plus: Next PyTorch release will support Apple GPUs so devs can train neural networks on their own laptops

    In brief Miscreants can easily steal someone else's identity by tricking live facial recognition software using deepfakes, according to a new report.

    Sensity AI, a startup focused on tackling identity fraud, carried out a series of pretend attacks. Engineers scanned the image of someone from an ID card, and mapped their likeness onto another person's face. Sensity then tested whether they could breach live facial recognition systems by tricking them into believing the pretend attacker is a real user.

    So-called "liveness tests" try to authenticate identities in real-time, relying on images or video streams from cameras like face recognition used to unlock mobile phones, for example. Nine out of ten vendors failed Sensity's live deepfake attacks.

    Continue reading
  • Lonestar plans to put datacenters in the Moon's lava tubes
    How? Founder tells The Register 'Robots… lots of robots'

    Imagine a future where racks of computer servers hum quietly in darkness below the surface of the Moon.

    Here is where some of the most important data is stored, to be left untouched for as long as can be. The idea sounds like something from science-fiction, but one startup that recently emerged from stealth is trying to turn it into a reality. Lonestar Data Holdings has a unique mission unlike any other cloud provider: to build datacenters on the Moon backing up the world's data.

    "It's inconceivable to me that we are keeping our most precious assets, our knowledge and our data, on Earth, where we're setting off bombs and burning things," Christopher Stott, founder and CEO of Lonestar, told The Register. "We need to put our assets in place off our planet, where we can keep it safe."

    Continue reading
  • Conti: Russian-backed rulers of Costa Rican hacktocracy?
    Also, Chinese IT admin jailed for deleting database, and the NSA promises no more backdoors

    In brief The notorious Russian-aligned Conti ransomware gang has upped the ante in its attack against Costa Rica, threatening to overthrow the government if it doesn't pay a $20 million ransom. 

    Costa Rican president Rodrigo Chaves said that the country is effectively at war with the gang, who in April infiltrated the government's computer systems, gaining a foothold in 27 agencies at various government levels. The US State Department has offered a $15 million reward leading to the capture of Conti's leaders, who it said have made more than $150 million from 1,000+ victims.

    Conti claimed this week that it has insiders in the Costa Rican government, the AP reported, warning that "We are determined to overthrow the government by means of a cyber attack, we have already shown you all the strength and power, you have introduced an emergency." 

    Continue reading

Biting the hand that feeds IT © 1998–2022