'No BS' web host Gandi lives up to half of its motto... Some customer data wiped out in storage server meltdown

0xdeadbeef


Updated Customers of web hosting outfit Gandi.net have been left less than impressed by its handling of a data-destroying storage crash.

The France-based hosting provider on Thursday disclosed it had lost some customer data after a ZFS storage box in Luxembourg broke down and had to be replaced using a backup. Efforts to restore the data, however, failed, and there were no snapshots available to recover from.

"The storage unit became unavailable, prompting an interruption in service for all PaaS and IaaS services using the disk associated with that unit," the Gandi team said. "The data import on the emergency machine was not possible due to a corruption of the meta-data that we are not aware of the cause of."

The biz went on to say it was conducting a full postmortem of the incident that would yield further details on what exactly went wrong. Its techies are still trying to recover the lost data, and thus far have had no luck.

While the loss of information without a viable backup is bad enough, customers are up in arms over the response from the business, which bills itself as a "no bullshit" outfit. Speaking to an aggrieved punter on Twitter, Gandi's web marketing and comms person Julie Pelloille offered the following answer:

The conversation descended into farce as Pelloille tried to defend the firm's position that its no BS strategy "does not exclude flaws." The response from the customer was as expected, saying: "It's not a flaw, it (sic) YOUR design."

Pelloille tried to lighten the mood by using a memorable moment from fantasy epic Game of Thrones.

This failed, natch. The Gandi staffer later apologized, saying they "truly regret[ted] posting it."

A quick browse through the company's documentation shows users able to create back-up copies of volumes on a Gandi Cloud server via Snapshots on a scheduled basis. Some customers had taken this as meaning that Gandi was taking care of those snapshots while they dealt with the likes of databases themselves.

Not so, according to boss Stephan Ramoin, who tweeted: "Andrea, sorry about that and the incident. If we led you to believe that you had nothing to do on your side when warned multiple times to make your back ups, then we'll have to make it clearer, and stop assuming that it's an industry wide knowledge."

Needless to say, punters were upset with the response and the host's "it could happen to anyone" defense.

"Sure, we can all make mistakes," wrote one user, "but don't blame YOUR OWN major fuckup and not sticking to your promises to any other company!"

We couldn't put it any better ourselves. ®

Updated on 10 January to add

Following publication of the article, Gandi made contact to say that around 300 customers were affected. A spokesperson told us:

"We now have some hope that we may recover the data but as we can't confirm it at the moment, customers who needed or need an immediate recovery should use their own backups, as was our initial recommendation."

And how to prevent this from happening again?

"We will certainly give a lot of thought to this question when we complete a full post mortem of the incident, but at this moment our teams are all still focused on restoring customer data. In particular, we'll be looking at what improvements can be made to our recovery time, our documentation, and our communications."

Similar topics


Other stories you might like

  • UK Home Secretary delays Autonomy founder extradition decision to mid-December

    Could be a Christmas surprise in store from Priti Patel

    Autonomy Trial Autonomy founder Mike Lynch's pending extradition to the US has been kicked into the long grass again by the UK Home Office.

    Lynch is wanted in the US to stand trial on 17 charges of fraud and false accounting. He is alleged to have defrauded Hewlett Packard investors over the sale of British software firm Autonomy in 2011.

    Continue reading
  • Want to buy your own piece of the Pi? No 'urgency' says Upton of the listing rumours

    A British success story... what happens next?

    Industry talk is continuing to circulate regarding a possible public listing of the UK makers of the diminutive Raspberry Pi computer.

    Over the weekend, The Telegraph reported that a spring listing could be in the offing, with a valuation of more than £370m.

    Pi boss, Eben Upton, described the newspaper's article as "interesting" in an email to The Register today, before repeating that "we're always looking at ways to fund the future growth of the business, but the $45m we raised in September has taken some of the urgency out of that."

    Continue reading
  • All change at JetBrains: Remote development now, new IDE previewed

    Security, collaboration, flexible working: Fleet does it all apparently

    JetBrains has introduced remote development for its range of IDEs as well as previewing a new IDE called Fleet, which will form the basis for fresh tools covering all major programming languages.

    JetBrains has a core IDE used for the IntelliJ IDEA Java tool as well other IDEs such as Android Studio, the official programming environment for Google Android, PyCharm for Python, Rider for C#, and so on. The IDEs run on the Java virtual machine (JVM) and are coded using Java and Kotlin, the latter being primarily a JVM language but with options for compiling to JavaScript or native code.

    Fleet is "both an IDE and a lightweight code editor," said the company in its product announcement, suggesting perhaps that it is feeling some pressure from the success of Microsoft's Visual Studio Code, which is an extensible code editor. Initial language support is for Java, Kotlin, Go, Python, Rust, and JavaScript, though other languages such as C# will follow. Again like VS Code, Fleet can run on a local machine or on a remote server. The new IDE uses technology developed for IntelliJ such as its code-processing engine for features such as code completion and refactoring.

    Continue reading

Biting the hand that feeds IT © 1998–2021