Heart Internet outage... three days and counting

Array with you! Biz still fixing failed drives

A number of Heart Internet customers remain unable to get online for three days thanks to a hosting failure.

On 28 January, the Nottingham-based web hosting biz explained in its status update page that one of its KVM hosts (KVMhost69) had suffered a disk failure, which had forced the file-system to mount in read-only mode.

"Servers on this host are currently in a shut-off state whilst we replace the failed drives in the Raid-10 array. We apologise for the inconvenience caused," it said.

Since Sunday, the firm has been busy replacing the failed disks in the array with new ones, which meant "minor data loss", and having to restore the affected virty servers from backups.

But in its latest post today, Heart said it was only halfway through the restorations for KVMhost69. "The remaining servers left to be restored are expected to be complete in the next 24 to 48 hours." However, the firm had estimated that same recovery time frame two days ago.

Meanwhile, a second KVM host appeared to have suffered connectivity issues. The team said in its latest update that all but a "small number of images" had been restored to "KVMhost21" from backup servers.

Folk took to Twitter to complain:

This is the longest our site has been down and money is being lost now @HeartInternet - so disappointed. How can I escalate this further as my bills don’t have downtime?

— Nick Betts (@NickBettsOnline) January 31, 2018

Hi @HeartInternet - any idea when the issue with the kvmhost hardware / connectivity will be sorted? We can’t access the backend of our site! 👎🏼

— Nick Betts (@NickBettsOnline) January 30, 2018

Hey guys, yes we're still experiencing issues. By all accounts this will be ongoing until Wednesday, maybe longer... :o(.

— E3 Digital (@EthreeDigital) January 30, 2018

Heart Internet was acquired by Host Europe Group in 2011, which in turn was bought by American domain registrar and hosting giant GoDaddy for €1.69bn (£1.48bn) in 2017.

The biz turned over £16.4m in revenue and £8.6m in profit before tax for the full-year 2016 – both flat on the previous year, according to its most recent P&L filing at Companies House.

A Heart mouthpiece told The Register: "This week two of Heart Internet's servers suffered failures which affected approximately 50 customers. Our engineers have fixed the issue and we expect all servers to be back online in the next 24 hours. We are reaching out to each affected customer to ensure they are aware of the issue and help with any problems. We apologise to our customers for any inconvenience caused." ®