OVH is yet to bring all customers affected by its Strasbourg data center fire back online – and the French cloud operator's CFO has described ongoing restoration efforts as a "real nightmare."
The fire took place on March 10 and destroyed the SBG2 hall of the Strasbourg data center, damaged SBG1 so badly it won't be revived, and led to a massive effort to clean salvageable kit so it could be installed in the remaining three data centers at Strasbourg, or moved to other OVH facilities.
Work is going slowly, according to the most recent tweet from OVH co-founder and chair Octave Klaba.
Update April,14 9pm— Octave Klaba (@olesovhcom) April 13, 2021
SBG1/E > Croix > SBG3
new 200 servers were shipped to Floor 5 which is now full. Tomorrow, we start filling Floor 3. Step by step. Yes, it’s slow. Real nightmare. pic.twitter.com/ylzMjzlEY3
In other tweets, Klaba said that all servers due to be moved from SBG1 to SBG4 have made the trip, with moves from SBG1 to SBG3 due this week.
"All servers are cleaned and tested, but it takes time to reassemble the racks... a huge Tetris," he wrote.
OVH no longer has a link to its incident report on the front page of its website, but an April 12 update revealed the following progress towards restoration:
- 92 per cent of servers for OVH Bare Metal Cloud have been delivered to customers
- 86 per cent of customer services running on Public Cloud instances have been replaced
- 99 per cent of Public Cloud Storage has been replaced
- 79 per cent of Hosted Private Clouds have been restored
- 84 per cent of Bare Metal Cloud Virtual Private Servers have been restored
OVH says burned data centre’s UPS, batteries, fuses in the hands of insurers and policeREAD MORE
The new update said cleaning each rack prior to reinstallation in a new data center takes seven hours, and that OVH staff are getting faster every day. However, France's renewed COVID-19 lockdown is not helping.
"We have had to source travel, accommodation and catering for all our employees who have come in to assist on site, while respecting government protocols, which of course made the operations more complex," the biz said.
Pandemic-related challenges, and the sheer complexity of the restoration effort, mean OVH has not offered a resolution date for the incident.
The company is also working on vouchers customers can use to pay for services and will not bill for services affected by the fire or replacement services ordered by customers. ®