Erasure coding startup springs forth from Silicon Fjord

Viking raider's longboat filled with erasure codes


Analysis Memoscale is a 6-person Norwegian startup, based in Trondheim, that has developed its own erasure coding (EC) technology. It says it's more efficient than classic erasure coding because it needs fewer hardware resources to run and enables higher storage capacity utilization.

All six staff members founded the company in 2015, but actual project work started in 2013 with error coding optimization for networking. Data storage was subsequently seen to be a better opportunity.

The CEO is Per Simonsen, previously the CEO at UniTimer AS. Katina Kralevska is a PhD student at Telematics-NTNU (Norwegian University of Science and Technology), which university was also attended by Simonsen. Kralevska's PhD thesis was entitled "Applied Erasure Coding in Networks and Distributed Storage" and submitted in August 2016. Software developer Kjetil Babington is also an NTNU graduate and spent time at UniTimer.

MemoScale's chairman is Eric Wold who is Head of Venture at NTNU Technology Transfer AS, and NTNU Technology Transfer AS is MemoScale's largest shareholder. NTNU professor Danilo Gligoroski is on MemoScale's board.

MemoScale is a company based on IP generated at NTNU by NTNU students, and it is being helped to become a viable business by NTNU technology-to-business transfer funding and expertise.

It claims that normal or classic EC (Reed-Solomon) has a 1.5x storage capacity overhead, whereas its technology only has a 1.2x overhead. Classic EC method facilitates a 13MB/sec recovery speed but Memoscale IP can provide a 34MB/sec speed.

Memoscale_tables

Memoscale erasure coding characteristics (Source: Philippe Nicolas)

Simplistically playing about with numbers we could say that, with a 500 drive array, classic EC with its 1.5x overhead leaves the capacity of 333 drives available for raw data, whereas Memoscale's own EC leaves 416.6 drives available – an approximate 83 drive advantage.

The recovery performance advantage in large arrays is significant, as disk failures and subsequent recoveries are a way of life in them. The combination of better storage efficiency and faster recovery should be appealing, as inefficient RAID protection gives way to EC and users/suppliers search for the least overhead and shortest recovery time commensurate with the protection levels they need.

Memoscale has four products:

  • Erasure coding C-library
  • Erasure coding plugin for Ceph
  • Erasure coding plugin for HDFS
  • Erasure coding plugin for SWIFT

The C-library product is said to be highly optimized for Intel and ARM processors, and Memoscale says it features:

  • Unmatched encode and decode speed
  • Support for multiple types of erasure codes
  • 3X faster recovery of drives and nodes using proprietary codes with reduced recovery traffic
  • Optional bit error detection and correction on reads

Memoscale has, it says, improved data scrubbing operations, as well as an interface for end-to-end checksum protection. Its software can support Reed-Solomon EC so users need not be locked in to Memoscale. It can process Reed-Solomon EC faster than other approaches such as Intel's EC library and Jerasure.

The Library and Ceph products are available now, with the HDFS and SWIFT plugins coming soon.

MemoScale has been granted around 10 million NOK from the Research Council of Norway and NTNU Discovery. More funding is being sought.

The company can sell its software to large Ceph users and is also looking to forge partnerships with storage suppliers. Kaminario has mentioned it has had discussions with Memoscale.

Simonsen says he is in the US once a month visiting potential partners. So watch out, a Viking raider is about; crossing the Atlantic again, with his metaphorical longboat filled with erasure codes and bent on replacing Reed-Solomon. ®


Other stories you might like

  • Talos names eight deadly sins in widely used industrial software
    Entire swaths of gear relies on vulnerability-laden Open Automation Software (OAS)

    A researcher at Cisco's Talos threat intelligence team found eight vulnerabilities in the Open Automation Software (OAS) platform that, if exploited, could enable a bad actor to access a device and run code on a targeted system.

    The OAS platform is widely used by a range of industrial enterprises, essentially facilitating the transfer of data within an IT environment between hardware and software and playing a central role in organizations' industrial Internet of Things (IIoT) efforts. It touches a range of devices, including PLCs and OPCs and IoT devices, as well as custom applications and APIs, databases and edge systems.

    Companies like Volvo, General Dynamics, JBT Aerotech and wind-turbine maker AES are among the users of the OAS platform.

    Continue reading
  • Despite global uncertainty, $500m hit doesn't rattle Nvidia execs
    CEO acknowledges impact of war, pandemic but says fundamentals ‘are really good’

    Nvidia is expecting a $500 million hit to its global datacenter and consumer business in the second quarter due to COVID lockdowns in China and Russia's invasion of Ukraine. Despite those and other macroeconomic concerns, executives are still optimistic about future prospects.

    "The full impact and duration of the war in Ukraine and COVID lockdowns in China is difficult to predict. However, the impact of our technology and our market opportunities remain unchanged," said Jensen Huang, Nvidia's CEO and co-founder, during the company's first-quarter earnings call.

    Those two statements might sound a little contradictory, including to some investors, particularly following the stock selloff yesterday after concerns over Russia and China prompted Nvidia to issue lower-than-expected guidance for second-quarter revenue.

    Continue reading
  • Another AI supercomputer from HPE: Champollion lands in France
    That's the second in a week following similar system in Munich also aimed at researchers

    HPE is lifting the lid on a new AI supercomputer – the second this week – aimed at building and training larger machine learning models to underpin research.

    Based at HPE's Center of Excellence in Grenoble, France, the new supercomputer is to be named Champollion after the French scholar who made advances in deciphering Egyptian hieroglyphs in the 19th century. It was built in partnership with Nvidia using AMD-based Apollo computer nodes fitted with Nvidia's A100 GPUs.

    Champollion brings together HPC and purpose-built AI technologies to train machine learning models at scale and unlock results faster, HPE said. HPE already provides HPC and AI resources from its Grenoble facilities for customers, and the broader research community to access, and said it plans to provide access to Champollion for scientists and engineers globally to accelerate testing of their AI models and research.

    Continue reading

Biting the hand that feeds IT © 1998–2022