Data centre outfit Interxion hit with outage at central London facility

Users complain: We weren't told that LON1 had done one

Updated European data centre operator Interxion suffered an outage at its central London campus last night, with no service from its LON1 data centre for several hours and users complaining the company was silent about what was happening.

In pre-COVID-times: Ely's Yard, behind the Old Truman Brewery building on Brick Lane that houses LON1

In pre-COVID times, clearly: Ely's Yard, behind the Old Truman Brewery building that houses LON1

The incident appears to have started shortly after 1800 UTC on Monday evening, with a loss of power at the LON1 site in the city's East End, which is conveniently located near to the financial district and the "Silicon Roundabout" tech hub in Shoreditch, according to Interxion.

Unfortunately for Interxion, built-in redundancy did not save it in this instance as the failure struck multiple power feeds into the building. Worse, it seems that the electronic switchgear designed to change the power over to an on-site generator also failed, and this prevented the company from switching between the mains supply and the backup generator supply.

A source whose employer is a member of the London Internet Exchange (LINX) informed The Register that there was "a flurry of emails on the member's mailing list covering the full spectrum from concerned (about what's going on and the possible extent) to angry (about lack of comms and inability to get hold of anyone from Interxion)."

LINX systems at the site appear not to have been affected, with the outage apparently affecting the entire first floor of the data centre site. It is not clear what may have caused multiple power feeds into the LON1 site to fail at the same time.

One user that was affected by the outage at Interxion's site was Vultr, a provider of cloud-based server infrastructure. Vultr's online server status notifications reported a partial outage in its London location starting at 1805 UTC, updating to show a complete outage by 1836 UTC.

By 2124 UTC, Vultr was reporting that "our data centre partner is continuing to work on restoring power to the site."

By this time, Interxion had apparently managed to restore some building systems and was attempting to restore power to the rest of the facility. It was 0136 UTC the following morning before Vultr was able to report that its entire London-based infrastructure was back online and operational.

Sargasso Networks, another Interxion customer, tweeted at 1938 UTC that it was "currently experiencing a major outage due to failure of multiple (allegedly resilient) power feeds at Interxion, over 90 minutes ago."

The company complained that "as Interxion have still failed to share any information and refuse to answer the telephone, we do not yet have an ETTR."

We contacted Interxion for a statement regarding the incident, and will update if we receive a response.

Interxion (pronounced "Interaction", apparently, a subject of some debate in the newsroom this morning) boasts of having facilities across 11 European countries and 13 metropolitan areas. It was acquired in 2020 by US data centre outfit Digital Realty in a deal said to be worth $8.4bn at the time.

We have asked the data centre outfit for comment and will update when it responds. ®

Updated to add

Interxion has been on touch to say: "At 1810 on Monday, 10th January, Interxion's Hanbury Street facility (LON1), experienced a critical power interruption, which impacted some of the mains control equipment and caused outages across the services in LON1.

"The problem was quickly identified and services started to come back online from 1945 until 2230 when the facility returned to operational status."

It added that it apologised "to all our customers and partners affected and for difficulties communicating during the outage."

"A full investigation is underway to determine the root cause of the interruption, the findings of which will be used to ensure an even more resilient infrastructure in the future. We remain proud of our reputation for global reliability and availability and have maintained five nines uptime over the past 14 years."

Similar topics

Other stories you might like

  • Microsoft's do-it-all IDE Visual Studio 2022 came out late last year. How good is it really?

    Top request from devs? A Linux version

    Review Visual Studio goes back a long way. Microsoft always had its own programming languages and tools, beginning with Microsoft Basic in 1975 and Microsoft C 1.0 in 1983.

    The Visual Studio idea came from two main sources. In the early days, Windows applications were coded and compiled using MS-DOS, and there was a MS-DOS IDE called Programmer's Workbench (PWB, first released 1989). The company also came up Visual Basic (VB, first released 1991), which unlike Microsoft C++ had a Windows IDE. Perhaps inspired by VB, Microsoft delivered Visual C++ 1.0 in 1993, replacing the little-used PWB. Visual Studio itself was introduced in 1997, though it was more of a bundle of different Windows development tools initially. The first Visual Studio to integrate C++ and Visual Basic (in .NET guise) development into the same IDE was Visual Studio .NET in 2002, 20 years ago, and this perhaps is the true ancestor of today's IDE.

    A big change in VS 2022, released November, is that it is the first version where the IDE itself runs as a 64-bit process. The advantage is that it has access to more than 4GB memory in the devenv process, this being the shell of the IDE, though of course it is still possible to compile 32-bit applications. The main benefit is for large solutions comprising hundreds of projects. Although a substantial change, it is transparent to developers and from what we can tell, has been a beneficial change.

    Continue reading
  • James Webb Space Telescope has arrived at its new home – an orbit almost a million miles from Earth

    Funnily enough, that's where we want to be right now, too

    The James Webb Space Telescope, the largest and most complex space observatory built by NASA, has reached its final destination: L2, the second Sun-Earth Lagrange point, an orbit located about a million miles away.

    Mission control sent instructions to fire the telescope's thrusters at 1400 EST (1900 UTC) on Monday. The small boost increased its speed by about 3.6 miles per hour to send it to L2, where it will orbit the Sun in line with Earth for the foreseeable future. It takes about 180 days to complete an L2 orbit, Amber Straughn, deputy project scientist for Webb Science Communications at NASA's Goddard Space Flight Center, said during a live briefing.

    "Webb, welcome home!" blurted NASA's Administrator Bill Nelson. "Congratulations to the team for all of their hard work ensuring Webb's safe arrival at L2 today. We're one step closer to uncovering the mysteries of the universe. And I can't wait to see Webb's first new views of the universe this summer."

    Continue reading
  • LG promises to make home appliance software upgradeable to take on new tasks

    Kids: empty the dishwasher! We can’t, Dad, it’s updating its OS to handle baked on grime from winter curries

    As the right to repair movement gathers pace, Korea’s LG has decided to make sure that its whitegoods can be upgraded.

    The company today announced a scheme called “Evolving Appliances For You.”

    The plan is sketchy: LG has outlined a scenario in which a customer who moves to a locale with climate markedly different to their previous home could use LG’s ThingQ app to upgrade their clothes dryer with new software that makes the appliance better suited to prevailing conditions and to the kind of fabrics you’d wear in a hotter or colder climes. The drier could also get new hardware to handle its new location. An image distributed by LG shows off the ability to change the tune a dryer plays after it finishes a load.

    Continue reading
  • IBM confirms new mainframe to arrive ‘late’ in first half of 2022

    Hybrid cloud is Big Blue's big bet, but big iron is predicted to bring a welcome revenue boost

    IBM has confirmed that a new model of its Z Series mainframes will arrive “late in the first half” of 2022 and emphasised the new device’s debut as a source of improved revenue for the company’s infrastructure business.

    CFO James Kavanaugh put the release on the roadmap during Big Blue’s Q4 2021 earnings call on Monday. The CFO suggested the new release will make a positive impact on IBM’s revenue, which came in at $16.7 billion for the quarter and $57.35bn for the year. The Q4 number was up 6.5 per cent year on year, the annual number was a $2.2bn jump.

    Kavanaugh mentioned the mainframe because revenue from the big iron was down four points in the quarter, a dip that Big Blue attributed to the fact that its last mainframe – the Z15 – emerged in 2019 and the sales cycle has naturally ebbed after eleven quarters of sales. But what a sales cycle it was: IBM says the Z15 has done better than its predecessor and seen shipments that can power more MIPS (Millions of Instructions Per Second) than in any previous program in the company’s history*.

    Continue reading
  • Earthquake halts operations at two of Toshiba's chip factories

    6.6-rated rumble joins fire, snow, plague, and trade war as source of recent semiconductor supply chain SNAFUs

    A 6.6 magnitude earthquake that hit southwestern Japan around 1:00 AM last Saturday has led to the closing of Toshiba’s Oita semiconductor plant.

    The Japan Meteorological Agency (JMA) said the 'quake may have caused significant shaking, making it difficult to walk unassisted and causing items on shelves to fall.

    The agency also warned that more tremors and earthquakes could occur in the immediate days following the seismic activity.

    Continue reading

Biting the hand that feeds IT © 1998–2022