Internet hub Telecity has failed to overcome the recent power fault responsible for frying computer hardware and denting UK internet traffic.
The data centre and colocation centre provider's plan was to employ a static transfer switch to transfer load to a mobile DR system installed especially in the Sovereign House car park, encompassing a seamless transition during a window of 23:00 and 02:00 local time Sunday evening and Monday morning.
But things were far from seamless, and resulted in yet more lost power, with engineers forced to leave customers running on unprotected mains supplies.
The attempted switch over was conducted by a combination of Telecity engineers and those from the manufacture of the Static Transfer Switch.
This is the second time Telecity’s team has not only failed in its attempts to solve the power problem, but also made matters worse – further outages were produced by emergency work last week.
Dotmailer sysadmin lead Jason Sanchania Tweeted:
At the time of the outage Telecity appeared to be handling 10 per cent of the UK’s internet traffic through Sovereign House alone. The outage saw web traffic for cloud providers (including Amazon and VoIP services running through Sovereign House) choked up, with a clear dent recorded in the UK’s internet traffic by the London Internet Exchange.
Disaster originally struck Sovereign House on the afternoon of 17 November, when engineers tried to conduct the second phase of a two-stage set of maintenance – testing the work done on Sovereign House’s power systems the week before.
However, engineers took the power out of phase producing a massive power spike that took out the B system UPS – a diesel-rotation uninterruptible power supply.
Sources tell us an A system UPS was unaffected, but customers whose systems were not connected to both – via a dual feed – saw their systems fried.
The Register has learned from disgruntled customers that their servers and disk arrays were completely destroyed.
Telecity did not respond to requests from The Register to comment or talk about the outage or its plans for repair. ®