Back up for a minute – Backblaze HD reliability stats show oldies can be goodies

Failure can be factored in when you're doing your sums, says vendor


Cloud storage and backup provider Backblaze has released a comprehensive report detailing reliability statistics for the hard drives it operated during the whole of 2021, with an interesting finding on its older kit.

The report leaves out boot drives (which are SSDs) and focuses on Backblaze's data drives, of which the firm had 203,168 under management as of December 31, 2021. These included various models sourced from all four major vendors: HGST, Seagate, Toshiba, and WDC. Of that figure, 409 drives were also excluded because they were used for evaluation or Backblaze did not have at least 60 examples of that model, leaving a total of 202,759 hard drives.

I would expect the larger capacity drives to ... fail more often as they age

Backblaze found that the drive model exhibiting the lowest failure rate was a 6TB Seagate drive (model ST6000DX000), with an annualised failure rate (AFR) of just 0.11 per cent. These also turned out to be the oldest in the Backblaze inventory, with an average age of over 80 months.

backblaze HD stats

Click to enlarge

The report also called out two drive models that are currently performing well but which were new to Backblaze in 2021, and so have not clocked up so many in-service hours. The 16TB WD drive cohort (model WUH721816ALE6L0) has an average age of 5.06 months and an AFR of 0.14 per cent, while the 16TB Toshiba drive cohort (model MG08ACA16TE) has an average age of 3.57 months and an AFR of 0.91 per cent.

Across all drive models, the AFR during 2021 was 1.01 per cent, slightly higher than the 0.93 per cent that Backblaze reported for 2020. However, both years were down from the rate of 1.83 per cent in 2019, which the firm said indicates that the downward trend is not an anomaly.

Looking back at all the different drive models aggregated together by manufacturer over the last three years appears to show that Seagate's failure rates have fluctuated the most on a quarter by quarter basis, while HGST drives consistently delivered a failure rate of under 1 per cent.

  Q1 2019 Q2 2019 Q3 2019 Q4 2019 Q1 2020 Q2 2020 Q3 2020 Q4 2020 Q1 2021 Q2 2021 Q3 2021 Q4 2021
HGST 0.41% 0.47% 0.61% 0.36% 0.37% 0.43% 0.30% 0.35% 0.47% 0.38% 0.31% 0.37%
Seagate 1.87% 2.22% 2.60% 2.72% 1.29% 0.90% 1.06% 1.21% 1.07% 1.41% 1.63% 1.48%
Toshiba 0.29% 0.00% 0.60% 1.06% 1.24% 1.09% 0.98% 0.67% 0.56% 0.75% 0.81% 0.92%
WDC 1.20%             0.16% 0.57% 0.46% 0.39% 0.32%
backblaze HD stats

The figures as a chart (image: Backblaze)

Meanwhile, the 2021 figures shows that annual failure rates for the larger drive models (12TB, 14TB, and 16TB) all come in below the annual average of 1.01 per cent across all drives, compared with 1.27 per cent for those of 10TB and below.

But the larger drives also tend to be the newer drives in service with Backblaze, and so are less likely to fail versus older drives. The oldest model of large drive has an average age of 33 months, while the newest model of 10TB and below has an average age of 44.9 months.

"Right now the larger capacity drives are all performing better than the average. But, as I noted all of the large capacity drive models are much younger than the smaller capacity drives we have," said cloud storage evangelist Andy Klein, adding: "I would expect the larger capacity drives to do the same, that is, fail more often as they age."

Backblaze also managed to solve an issue it had previously reported with 14TB Seagate drives (model ST14000NM0138) failing at a higher than expected rate.

It had the failed drives examined by fault analysis specialists and a simple decision to upgrade the firmware of all the units still operating resulted in the quarterly failure rate dropping from 6.29 per cent in Q3 to 4.66 per cent in Q4, stabilising the rapid rise in failures, Backblaze said.

Price matters

If any readers are wondering why Backblaze doesn't just focus on the drives that are most reliable, that isn't the way it works, because price is always a factor when procuring storage, Klein explained.

"Our architecture is built assuming drives will fail. We have multiple layers of protection to keep data safe when that happens. This allows us the flexibility to purchase drives based on price as long as they perform reasonably well in our environment," he said.

In addition, when there is a need to replace failed drives, the firm may not be able to purchase the same model again.

Backblaze said that its Vault storage architecture revolves around mixing and matching different drive models. A Vault is apparently composed of 60 tomes, with each tome made up of 20 drives. Each tome is made up of a single drive model, but the tomes making up an entire Vault may represent different drive models, and even different drive sizes. This allows the firm to be less reliant on any particular drive model.

The Backblaze report also includes a table of annualised failure rate figures for every drive model it had in production use as of December 31, 2021, covering a reporting period that started way back in March 2013.

backblaze HD stats

Click to enlarge

According to Backblaze, the lifetime AFR for all the drives in the table is 1.4 per cent, and continues to go down year after year. At the end of 2020, the comparable figure was 1.54 per cent and a year earlier the figure stood at 1.62 per cent.

However, Backblaze cautions that some of the drives in the table have a fairly wide confidence interval (greater than 0.5) which implies that there is not enough data about that model of drive's performance to have a reasonable level of confidence in the AFR listed, either because the drives are too new or there are too few of that model deployed.

Backblaze will be disclosing the annual failure rates for its SSD drives in a separate post sometime in the next few weeks. The hard drive data is also available to download from Backblaze for anyone who wishes to analyse the figures for themselves. ®

Similar topics

Broader topics

Narrower topics


Other stories you might like

  • Experts: AI should be recognized as inventors in patent law
    Plus: Police release deepfake of murdered teen in cold case, and more

    In-brief Governments around the world should pass intellectual property laws that grant rights to AI systems, two academics at the University of New South Wales in Australia argued.

    Alexandra George, and Toby Walsh, professors of law and AI, respectively, believe failing to recognize machines as inventors could have long-lasting impacts on economies and societies. 

    "If courts and governments decide that AI-made inventions cannot be patented, the implications could be huge," they wrote in a comment article published in Nature. "Funders and businesses would be less incentivized to pursue useful research using AI inventors when a return on their investment could be limited. Society could miss out on the development of worthwhile and life-saving inventions."

    Continue reading
  • Declassified and released: More secret files on US govt's emergency doomsday powers
    Nuke incoming? Quick break out the plans for rationing, censorship, property seizures, and more

    More papers describing the orders and messages the US President can issue in the event of apocalyptic crises, such as a devastating nuclear attack, have been declassified and released for all to see.

    These government files are part of a larger collection of records that discuss the nature, reach, and use of secret Presidential Emergency Action Documents: these are executive orders, announcements, and statements to Congress that are all ready to sign and send out as soon as a doomsday scenario occurs. PEADs are supposed to give America's commander-in-chief immediate extraordinary powers to overcome extraordinary events.

    PEADs have never been declassified or revealed before. They remain hush-hush, and their exact details are not publicly known.

    Continue reading
  • Stolen university credentials up for sale by Russian crooks, FBI warns
    Forget dark-web souks, thousands of these are already being traded on public bazaars

    Russian crooks are selling network credentials and virtual private network access for a "multitude" of US universities and colleges on criminal marketplaces, according to the FBI.

    According to a warning issued on Thursday, these stolen credentials sell for thousands of dollars on both dark web and public internet forums, and could lead to subsequent cyberattacks against individual employees or the schools themselves.

    "The exposure of usernames and passwords can lead to brute force credential stuffing computer network attacks, whereby attackers attempt logins across various internet sites or exploit them for subsequent cyber attacks as criminal actors take advantage of users recycling the same credentials across multiple accounts, internet sites, and services," the Feds' alert [PDF] said.

    Continue reading

Biting the hand that feeds IT © 1998–2022