Comment An academic paper claims flash could be better than disk for archiving. So just how did this unlikely result come about?
The SSRC* paper, An Economic Perspective of Disk vs. Flash Media in Archival Storage, was published earlier this year, at the 22nd IEEE International Symposium on Modelling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS 2014).
In it, lead author Preeti Gupta and her colleagues found that "archives based upon alternative media [such as flash] are surprisingly cost competitive [when compared with] archives based upon traditional disk media over the long-term".
They developed algorithms to test the costs of archival storage over a period of years using disk and flash. The paper starts off by saying archiving often resulted in storing data with infrequent access, making tape a good fit even though it has a long access latency.
Recently though, data mining of archives has developed, which requires faster access and so making disk preferential over tape. Gupta and her colleagues looked at whether flash was as good as disk, or even better in this low latency archive situation.
Long-term storage costs include media replacement for worn-out drives, and the industry expects replacement disk products to have a higher capacity than the replaced drives, and with a lower per-GB cost.
A main influence on replacement disk costs is the Kryder rate, "the annual proportional change in bit density and thus, we assume, decrease in per-byte cost (so a rate at 0.2 means next year’s per-byte cost would be 80 per cent of this year’s per-byte cost).
To make it clearer "per-byte storage costs decrease as bit density increases at the Kryder rate (annual storage density growth rate)."
The Kryder rate is slowing due to the dramatic increase in industry costs needed for a jump to the next recording technology, HAMR (Heat-Assisted Magnetic Recording). "Compared with pre-2010 projections, Figure 1 shows that per-byte disk cost is now seven times more expensive, and in six years would be around 200 times more expensive."
Fig 1. Kryder rate slow down
The researchers write: "The transitions to the next disk recording technology (HAMR) and its probable successor (Bit-Patterned Media) turn out to be vastly more difficult and expensive than expected, delaying further bit density improvements and thus decreasing the Kryder rate."
The archival access pattern is described as write once, read rarely, overwrite rarely. Flash is more expensive per-GB to buy than disk but is cheaper to use, needing less power, space and cooling.
However, as its geometry scales down its endurance worsens. Flash drive controllers, currently mostly optimised for performance, can be optimised for endurance instead and solve that problem. The researcher suggest flash drives could be made with stronger insulation between cells, at little extra cost, to increase the data retention period.