CenturyLink L3 outage knocks out web giants and 3.5% of all internet traffic

Cloudflare fingers intertwined BGP and Flowspec SNAFUs


Internet backbone operator CenturyLink has experienced an outage that degraded performance of major web companies around the world.

CenturyLink acknowledged the outage and posted basic information about the incident.

CloudFlare offered much more detail, explaining that the outage impacted the L3 infrastructure that CenturyLink acquired in 2017 and was felt in at least 48 cities.

Border Gateway Protocol (BGP) was suggested as the cause. "Starting at 10:04 UTC, there were a significant number of BGP updates," CloudFlare's post said. "Under normal conditions, the Internet sees about 1.5MBs-2MBs of BGP updates every 15 minutes. At the start of the incident, the number of BGP updates spiked to more than 26MBs of BGP updates per 15 minute period and stayed elevated throughout the incident."

A man passed out

Fat-fingered Level 3 techie reduces internet to level zero: Glitch knocks out connections

READ MORE

CloudFlare's speculated that a bad Flowspec rule was the source of the incident and may have prevented successful BGP propagation, which just meant more BGP traffic.

Whatever the cause, the incident meant that CloudFlare was briefly not at its best and so, therefore, were some large customers including Discord, Twitter, Xbox Live, PlayStation network, Amazon and many others.

The Register cannot, however, find outage reports from many the impacted sites, suggesting that alternative routes to L3 may have been slow picking up the slack. Companies reliant on CenturyLink alone fared rather worse, but at least did so at a slowish time of the week. ®


Biting the hand that feeds IT © 1998–2020