Boffins map Netflix's Open Connect CDN

Six continents, 233 locations, thousands of servers

Brit boffins have peeled back the covers of how Netflix has built its CDN, by requesting movies from all over the world and working out what the responses told them about the hosts.

The research incidentally explains why the entertainment biz wants the Internet exchange (IXP) industry to trim its costs: in the USA, IXPs deliver around 1,600 Gbps of Netflix traffic.

Netflix has talked up its global deployment, but hasn't gone public with the kind of detail the Queen Mary University of London researchers offer in this paper at Arxiv.

The five researchers turned up servers in six continents and 233 individual locations, along the way turning up some surprises in traffic patterns.

While nobody would doubt the assertion that the USA is still where the action is for Netflix traffic, the researchers reckon the number-two spot is held by Mexico, followed by the UK, Canada and Brazil.

To get their data, the researchers started by firing off manual requests for videos, localising the requests using the Hola browser extension to work out the company's naming conventions (it's not surprising that Netflix is, as the paper says, “deterministic” in server naming: the alternative in a network that size would be chaos).

Hence, and told the researchers they were talking to an IPv6 server; “lagg0” is one of various names used to distinguish the type of network connection; “c002” says there are two servers in the location delivering the content; lhr001 identifies the nearest airport (in this case, London Heathrow); the ISP is BT (“bt.isp”); “ix” reveals that the Netflix location in question is inside an Internet exchange; and is the common domain for all servers.

That structure made it feasible to use agents to automate the enumeration of the infrastructure.

“We implemented a crawler, which generated DNS names following the pattern outlined above and attempted to resolve those generated domains names to IP addresses. The crawler was fed with lists of known countries, airport codes and ISPs. We curated these lists manually, consulting publicly available lists of countries and airport codes”, the paper continues.

And here's what the network looks like at a global scale:

Netflix global server locations

Where the Netflix servers are.
Image: Open Connect Everywhere: A Glimpse at the
Internet Ecosystem through the Lens of the Netflix CDN, Böttger et al

Since the researchers don't have access to Netflix's “real” flows, they estimated from each server by firing pings at each server, and “tracking the evolution of the ID values in the ICMP answers”.

Since Netflix is a FreeBSD-everywhere, they write, the packet IDs serve as a handy (if approximate) proxy for traffic loads.

Their results on a country-by-country basis are below.

Netflix global traffic

Internet exchanges - absolutely vital to the Netflix CDN,
Image: Open Connect Everywhere: A Glimpse at the
Internet Ecosystem through the Lens of the Netflix CDN, Böttger et al

It's no surprise that in the USA, the world's most mature peering market, Internet exchanges ship the overwhelming bulk of Netflix traffic. Where the Internet is dominated by national-scale incumbents, IXs deliver less traffic (Brazil, the UK and Australia, for example), while in Mexico and Canada, Netflix traffic emanates entirely from ISPs. ®

Similar topics

Other stories you might like

  • Google sours on legacy G Suite freeloaders, demands fee or flee

    Free incarnation of online app package, which became Workplace, is going away

    Google has served eviction notices to its legacy G Suite squatters: the free service will no longer be available in four months and existing users can either pay for a Google Workspace subscription or export their data and take their not particularly valuable businesses elsewhere.

    "If you have the G Suite legacy free edition, you need to upgrade to a paid Google Workspace subscription to keep your services," the company said in a recently revised support document. "The G Suite legacy free edition will no longer be available starting May 1, 2022."

    Continue reading
  • SpaceX Starlink sat streaks now present in nearly a fifth of all astronomical images snapped by Caltech telescope

    Annoying, maybe – but totally ruining this science, maybe not

    SpaceX’s Starlink satellites appear in about a fifth of all images snapped by the Zwicky Transient Facility (ZTF), a camera attached to the Samuel Oschin Telescope in California, which is used by astronomers to study supernovae, gamma ray bursts, asteroids, and suchlike.

    A study led by Przemek Mróz, a former postdoctoral scholar at the California Institute of Technology (Caltech) and now a researcher at the University of Warsaw in Poland, analysed the current and future effects of Starlink satellites on the ZTF. The telescope and camera are housed at the Palomar Observatory, which is operated by Caltech.

    The team of astronomers found 5,301 streaks leftover from the moving satellites in images taken by the instrument between November 2019 and September 2021, according to their paper on the subject, published in the Astrophysical Journal Letters this week.

    Continue reading
  • AI tool finds hundreds of genes related to human motor neuron disease

    Breakthrough could lead to development of drugs to target illness

    A machine-learning algorithm has helped scientists find 690 human genes associated with a higher risk of developing motor neuron disease, according to research published in Cell this week.

    Neuronal cells in the central nervous system and brain break down and die in people with motor neuron disease, like amyotrophic lateral sclerosis (ALS) more commonly known as Lou Gehrig's disease, named after the baseball player who developed it. They lose control over their bodies, and as the disease progresses patients become completely paralyzed. There is currently no verified cure for ALS.

    Motor neuron disease typically affects people in old age and its causes are unknown. Johnathan Cooper-Knock, a clinical lecturer at the University of Sheffield in England and leader of Project MinE, an ambitious effort to perform whole genome sequencing of ALS, believes that understanding how genes affect cellular function could help scientists develop new drugs to treat the disease.

    Continue reading

Biting the hand that feeds IT © 1998–2022