Flash data storage: Knocking TAPE off the archiving top spot

Could this be the FLAPE of things to come?


It was a bombshell when Facebook's Jason Taylor said he would like to use flash solid-state storage as an archive medium, but his reasons made perfect sense.

Facebook users had lots of photos stored in their albums and rarely accessed them. But when they did want to look at them they wanted them to come up straight away, whether the photos were recent or five years old.

Only a particular form of flash storage could, in theory, combine the attributes Taylor wanted: low-cost, long-term, reliable storage and fast access.

The two current archive media types, tape and disk, rely on being offline when not being accessed and thus consuming next to no power. As soon as an archive medium is accessed it uses power, and electricity cost is a huge concern when you are storing many petabytes of data, heading towards exabyte levels, over many years.

A tape archive is a library with few drives and hundreds or thousands of slots for offline tape cartridges. Robot mechanisms identify where a cartridge is located when a file is needed and deliver it to a drive to be mounted and spun to the right location on the wound tape so the file can be accessed.

Make it a double

Tape is intrinsically the cheapest archive medium. Unlike disk drives, tape cartridges have no costly embedded drive or motor. They hold data safely for many years or even decades and are robust. They can also compress data and that can double or triple the cartridge's effective capacity.

A tape library runs cool as it has only a few drives that consume power and its robot mechanisms don't use much power either. This means that it uses a fraction of the power and cooling needed for an equivalent amount of data in an online disk drive array.

A tape archive still has a much lower total cost of ownership

The library takes up data-centre space, of course, and that is a cost but compared with a disk-based archive, a tape archive still has a much lower total cost of ownership.

As a result the idea of using disk as an archive was laughable until recently. Two advances, however, have changed the maths: deduplication and spinning down disks when not in use.

Deduplication: Much better than compression at removing repeated strings of bytes in data, deduplication has allowed tape to be replaced by disk for backing up and storing data for short-term protection needs such as recovering lost files or replacing corrupted data.

Deduplication can achieve as much as a 5:1 data reduction ratio with backup or VDI data (less so with images and structured data). That, together with disk’s faster data access*, was more than enough to cause wholesale replacement of tape by disk for backup.

Spinning down disks when not in use: Spinning up the disk and accessing data only when needed is a much quicker process than finding a tape in a library, robotically moving it to a drive, mounting it and streaming it to the right location.

Even though disk capacity has just reached 8TB, and 10TB is in prospect with advances like shingled magnetic recording, and every capacity increase lowers the cost/GB of disk-stored data, tape is increasing its density faster, with 154TB cartridges demonstrated by IBM.

The Wikibon consultancy declares tape "areal density is growing at approximately 30 per cent versus disk, which is growing at only 9.6 per cent."

A non-spinning disk needs no power and that means the cost of deduplicating and spinning down disk is approaching tape storage costs. It is still not as cost-efficient as tape for data archiving, or as reliable, but in cases where archive data access speed is a high priority, disk is beginning to be used.

Its users, though, would like a cheaper way of archiving their data without losing disk’s speed advantage. And, they would say, spun-down disk is not fast enough for data access. It may be better than tape but that is like comparing walking to crawling when what you really want is a medium that can sprint.

Some archive use cases have settled on Blu-ray optical disks as the archive medium but we understand this is a minority and not a mainstream archive medium. Here's a Facebook example.

Next page: Where flash shines

Other stories you might like

  • Experts: AI should be recognized as inventors in patent law
    Plus: Police release deepfake of murdered teen in cold case, and more

    In-brief Governments around the world should pass intellectual property laws that grant rights to AI systems, two academics at the University of New South Wales in Australia argued.

    Alexandra George, and Toby Walsh, professors of law and AI, respectively, believe failing to recognize machines as inventors could have long-lasting impacts on economies and societies. 

    "If courts and governments decide that AI-made inventions cannot be patented, the implications could be huge," they wrote in a comment article published in Nature. "Funders and businesses would be less incentivized to pursue useful research using AI inventors when a return on their investment could be limited. Society could miss out on the development of worthwhile and life-saving inventions."

    Continue reading
  • Declassified and released: More secret files on US govt's emergency doomsday powers
    Nuke incoming? Quick break out the plans for rationing, censorship, property seizures, and more

    More papers describing the orders and messages the US President can issue in the event of apocalyptic crises, such as a devastating nuclear attack, have been declassified and released for all to see.

    These government files are part of a larger collection of records that discuss the nature, reach, and use of secret Presidential Emergency Action Documents: these are executive orders, announcements, and statements to Congress that are all ready to sign and send out as soon as a doomsday scenario occurs. PEADs are supposed to give America's commander-in-chief immediate extraordinary powers to overcome extraordinary events.

    PEADs have never been declassified or revealed before. They remain hush-hush, and their exact details are not publicly known.

    Continue reading
  • Stolen university credentials up for sale by Russian crooks, FBI warns
    Forget dark-web souks, thousands of these are already being traded on public bazaars

    Russian crooks are selling network credentials and virtual private network access for a "multitude" of US universities and colleges on criminal marketplaces, according to the FBI.

    According to a warning issued on Thursday, these stolen credentials sell for thousands of dollars on both dark web and public internet forums, and could lead to subsequent cyberattacks against individual employees or the schools themselves.

    "The exposure of usernames and passwords can lead to brute force credential stuffing computer network attacks, whereby attackers attempt logins across various internet sites or exploit them for subsequent cyber attacks as criminal actors take advantage of users recycling the same credentials across multiple accounts, internet sites, and services," the Feds' alert [PDF] said.

    Continue reading

Biting the hand that feeds IT © 1998–2022