HostingUK and its big brother, iomart, are still struggling to restore online services more than 12 hours after a bit barn blackout.
A fibre cable break took down the St Asaph data centre in Wales last night and, judging by the howling on social media, pretty much all of HostingUK.
Users began reporting problems from 2055 UK time on 31 May, finding sites inaccessible and unable to contact either HostingUK or iomart for help.
@hostinguknet you having an outage? I can’t access my sites.— Danny Whisky (@whisky_duck) May 31, 2018
HostingUK tweeted that something was amiss at 2116, although their status page showed their engineers became aware that A Bad Thing had happened at 2047.
By 2119 the extent of the problem was becoming clear, with engineers suspecting that two converging fibre paths had been severed, leaving the St Asaph data centre totally isolated.
At the time HostingUK reckoned that recovery would take anywhere between two and 12 hours. It would seem the latter estimate was closer to the mark.
What followed was a Pythonesque** scene of Zayo (the underlying provider) engineers playing a game of hunt-the-break, ending up in a field at 0554 this morning, after five hours of searching following a tech declaring "yup – the cable's busted", to find a trencher had put a blade through the unfortunate fibre.
Big daddy iomart weighed in with a tweet at 2138, directing affected users to the HostingUK status page (via an IP address, no less).
We are aware of customers of @hostinguknet experiencing connectivity issues. Please see their status page for more information http://184.108.40.206/hostinguk/hostinguk-system-status/ We are working to resolve the issue as quickly as possible.— iomart (@iomart) May 31, 2018
To their credit, HostingUK have continued to respond to wailing customers via Twitter and keep their status page updated as the on-site crews stumble around in the countryside. However, that responsiveness is of little comfort to users with online businesses that depend on, er, being online.
Eagle-eyed readers will, of course, be aware that iomart's infrastructure is no stranger to dropping over. The last unscheduled cable break knocked much of the network offline just over two months ago.
HostingUK and its St Asaph data centre were acquired by iomart in a 2012 deal worth £1.5m in a spending spree that also saw Manchester-based Melbourne Server Hosting consumed amongst others.
Customers waking up to websites offline while engineers frantically dig up fields will be wishing some of that cash had been spent on beefing up the infrastructure.
The Register has contacted HostingUK, iomart and infrastructure provider Zayo for comment and will update if any response is forthcoming. ®
* Totally Inept Trenching Stirs Users' Passions
** Resembling the absurd humour of Monty Python's Flying Circus, a much-loved British sketch comedy show of the early 1970s