New storage upstart Versity offers S3 object storage interface

As well as multi-threaded and SAM-QFS-based archiving software


Versity is an archiving software startup using multi-threaded SAM-QFS.

It was founded in March 2011 by CEO Bruce Gilpin, with a VC background, and CTO Harriet Coverston. She is the tech brains of the outfit and worked at LSC (Large Storage Configurations) from 1986, which developed QFS (Quick File System).

QFS grouped disk drives and provided a file system for them. This software was bought by Sun, where Coverston worked and then found a home in Oracle when Sun was bought by Larry's company.

At Sun QFS was paired with Storage and Archive Manager (SAM), which manages disk and tape bulk storage, and so became SAM-QFS. Sun gave it to the OpenSolaris project in March 2008. It is a hierarchical storage management product (HSM) with cold files copied to tape (IBM, LTO and Oracle formats) from the front-end disk store, as a background operation, and retrieved to disk when needed.

Oracle Hierarchical Storage Manager is basically SAM-QFS.

Versity was self-funded, and run in a penny-pinching and deferred bills payment way for two years, until an undisclosed A-round of funding from Cray in July 2013. Cray is a distribution partner in the HPC market. Versity Storage Manager (VSM), its software product, was released in April 2014.

The software is used by between 10 and 20 customers, including Ovation Data with more than 250PB managed. A million dollar contract was signed by Versity last year.

Gilpin told an IT Press Tour that VSM is open source software and users subscribe to use it. Specifically VSM is a proprietary open core product which legally incorporates SAM-QFS source code, licensed to Versity under the CDDL.

Its architectural features include these:

  • Purpose-built archiving file system delivers industry-leading performance
  • Policy-based data management application delivers automated storage tiering for continuous storage optimization
  • Based on open source SAM-QFS technology installed at hundreds of customer sites worldwide
  • Writes to open tar file format to minimize vendor specific dependencies
  • <Li<Standard POSIX file system implementation supports a broad range of applications with no customization
VSM_scheme

The software is said to be fast:

  • Tunable system parameters adjust to any application, network type or storage technology
  • Archiver moves files at nearly the raw speed of storage hardware
  • Metadata can be stored separately to maximize throughput or interspersed with data to accommodate transactional environments
  • Variable DAU (Disk Allocation Unit) feature excels in both large and small file performance and can be tuned to support differing workloads

VSM is not object storage and is not a parallel file system. It is archival storage for organisations that do not want to write to an object API, but whch want to retain their existing POSIX interfaces.

Development

Versity is releasing an S3 interface for the backend store, which could be a public cloud or an on-premises S3-using object store. There is a customer for this and the hardware supplier, Gilpin says: "is one of on-premises object storage vendors who sell an appliance."

Why is Versity supporting an object interface? It says "Because the web scale data centres built a better mousetrap (object storage) but the rest of us live in the POSIX world and don’t have plans to move!"

Versity is adding a second object storage interface (think S-like minus load-balancing) to integrate VSM with an on-premises object store in a way that is faster and simpler than with S3. We hear that the resulting system will be one third of Amazon's S3 storage cost.

We suspect the backend object store supplier could be Cloudian.

Multi-threading

Many files are packaged into a single TAR file, described as traffic shaping, and this is written to the archive backing store as a single entity. The TAR file size can be set by policies.

VSM then multi-threads IO with threads running on one or several processor cores. Several of its developers are Lustre people and know parallel file systems. Gilpin provided anecdotal evidence of speed, saying VSM could write to a backend store at 1.8GB/sec using a $5,000 Dell server fitted with, he thought, a pair of 10GbitE NICs. The reading (staging as he called it) speed was 2.1GB/sec.

This is similar to the techniques used by DataCore's Parallel Server technology.

Gilpin reckons VSM is faster than Avere's filer, object store and cloud access accelerator products.

Scale-out filesystem

Versity is developing a scale-out POSIX file system that can cope with up to 1 trillion files. It will be paired with an object store and has been 8 months in development. The company hopes to have a demonstrable alpha version of the code by October.

It will be a peer-to-peer system with no master node, and have byte-range locking and a B-tree data structure.

Gilpin said it is specifically for archiving and ignores POSIX in byte ranges where there is no contention, meaning no file locking by default. When there is contention it slows down to "NFS" like speed. Metadata needs to be on the fastest-possible block storage.

This will be Versity's next-generation product.

Gilpin also said Versity is thinking about better ways to take in files from Lustre systems, possibly even supporting the Lustre file format directly.

Competition and comment

The competition is IBM 9With (LTFS LE, TSM, and HPSS), Oracle (SAM-QFS, Oracle HSM), SGI (DSM) and Quantum (StorNext). He said Quantum is probably its strongest product competitor but IBM and Oracle have stronger channels and customer relationships.

SpectraLogic was mentioned as a Versity partner.

El Reg thinks you can't argue with a million dollar contract and an inward investment from Cray; this is a serious company in its archiving sphere, and VSM's multi-threading speed looks impressive.

The scale-out development will answer the limited scalability point object storage suppliers always make about file systems. The S3 backend interface makes VSM an object storage gateway for POSIX application users. The additional object store API will help there too. And the scale-out file system development looks exciting. Watch out for news on this in four months or so. ®

Similar topics

Broader topics


Other stories you might like

  • Venezuelan cardiologist charged with designing and selling ransomware
    If his surgery was as bad as his opsec, this chap has caused a lot of trouble

    The US Attorney’s Office has charged a 55-year-old cardiologist with creating and selling ransomware and profiting from revenue-share agreements with criminals who deployed his product.

    A complaint [PDF] filed on May 16th in the US District Court, Eastern District of New York, alleges that Moises Luis Zagala Gonzalez – aka “Nosophoros,” “Aesculapius” and “Nebuchadnezzar” – created a ransomware builder known as “Thanos”, and ransomware named “Jigsaw v. 2”.

    The self-taught coder and qualified cardiologist advertised the ransomware in dark corners of the web, then licensed it ransomware to crooks for either $500 or $800 a month. He also ran an affiliate network that offered the chance to run Thanos to build custom ransomware, in return for a share of profits.

    Continue reading
  • China reveals its top five sources of online fraud
    'Brushing' tops the list, as quantity of forbidden content continue to rise

    China’s Ministry of Public Security has revealed the five most prevalent types of fraud perpetrated online or by phone.

    The e-commerce scam known as “brushing” topped the list and accounted for around a third of all internet fraud activity in China. Brushing sees victims lured into making payment for goods that may not be delivered, or are only delivered after buyers are asked to perform several other online tasks that may include downloading dodgy apps and/or establishing e-commerce profiles. Victims can find themselves being asked to pay more than the original price for goods, or denied promised rebates.

    Brushing has also seen e-commerce providers send victims small items they never ordered, using profiles victims did not create or control. Dodgy vendors use that tactic to then write themselves glowing product reviews that increase their visibility on marketplace platforms.

    Continue reading
  • Oracle really does owe HPE $3b after Supreme Court snub
    Appeal petition as doomed as the Itanic chips at the heart of decade-long drama

    The US Supreme Court on Monday declined to hear Oracle's appeal to overturn a ruling ordering the IT giant to pay $3 billion in damages for violating a decades-old contract agreement.

    In June 2011, back when HPE had not yet split from HP, the biz sued Oracle for refusing to add Itanium support to its database software. HP alleged Big Red had violated a contract agreement by not doing so, though Oracle claimed it explicitly refused requests to support Intel's Itanium processors at the time.

    A lengthy legal battle ensued. Oracle was ordered to cough up $3 billion in damages in a jury trial, and appealed the decision all the way to the highest judges in America. Now, the Supreme Court has declined its petition.

    Continue reading

Biting the hand that feeds IT © 1998–2022