Off-Prem

SaaS

BBC makes switch to AWS, serverless for new website architecture, observers grumble about the HTML

News aggregator says it's 'way more complicated and much harder to parse'


Updated The BBC website, the sixth most popular in the UK, has mostly migrated from the broadcaster's bit barns to Amazon Web Services (AWS) with around half the site now rendered using AWS Lambda, a serverless platform.

"Until recently much of the BBC website was written in PHP and hosted on two data centres near London," Matthew Clark, head of architecture, said lately. "Almost every part has been rebuilt on the cloud."

PHP runs fine in the cloud, but this is not a matter of lift and shift. Instead, the BBC team devised a new architecture based on serverless computing. It also endeavoured to combine what used to be several sites – such as News, Sport, and so on – into one, though Clark said the World Service, iPlayer video, and the radio site BBC Sounds remain separate.

The rest have been combined into a new thing called WebCore. "By focussing on creating one site, rather than several, we're seeing significant improvements in performance, reliability, and SEO," said Clark.

Web traffic initially hits a Global Traffic Manager (GTM), an in-house solution based on the Nginx web server and running partly on-premises (showing that the BBC has not entirely ditched its data centres) and partly on AWS. GTM handles "tens of thousands of requests a second," said Clark. A second layer on AWS handles caching and routing, before hitting functions running on AWS Lambda, which perform server-side rendering (SSR) of dynamic content using React, a JavaScript framework.

Server-side rendering means the browser gets a page ready to view without having to do a lot of work, and thus it should appear instantly, though it increases the burden on the server – caching mitigates this, we note. Walmart engineer Alex Grigoryan, who also oversaw a migration to SSR, tested SSR vs client-side rendering (CSR) and said: "When we did A/B tests on SSR vs CSR... our numbers showed better engagement from the customer with rendering early," though he noted increased server load as a major disadvantage.

In the BBC's case, Lambda is used, which is able to auto-scale on demand. "About 2,000 lambdas run every second to create the BBC website; a number that we expect to grow," said Clark. He added that Lambda scales better than VMs on the AWS Elastic Compute Cloud (EC2), saying that "our traffic levels can rocket in an instant; Lambda can handle this in a way that EC2 auto-scaling cannot."

Another aspect of the BBC site is the logic that goes into requesting content, which Clark calls the "business layer". Content is provided to the web rendering layer via a REST API, and a solution called Fast Agnostic Business Layer "allows different teams to create their own business logic," he said, so that different requirements are met while still sharing the same system for things like access control and caching. Clark didn't say much about how the content itself is stored, though he promised to return to this topic in future posts.

The WebCore platform uses CI/CD to enable rapid iteration, and Clark showed an example monthly report showing 110 releases or around three per day. Builds take around 3.5 minutes, and the average time from a pull request (when new code is merged) to running it in production was one day and 23 minutes, in this particular month. On average 67 per cent of pull requests were actually merged into the code.

A small section of the HTML delivered for a news article today on the BBC site. A news aggregator says it is much harder to parse than before

Great work? Comments on Hacker News show that opinions vary. "Running a site the size of the BBC on Lambda is nothing short of an exuberant waste of a government-subsidized budget, it's absolutely crazy. Lambda VM time has a massive markup compared to regular compute... IMHO this is the epitome of serverless gone wrong," said one.

Another comment from John Leach, who runs a headline aggregation site called News Sniffer, said that the generated HTML is not easy to analyze. "I run the News Sniffer project which has to parse BBC News pages and I knew about this rollout a few weeks ago when the HTML all changed format completely and my parsers broke. As a side note, the new HTML is way more complicated and much harder to parse than before – I know the aim isn't to help parsing for content, but I was still saddened to see how it's ended up."

There is also curiosity about unanswered questions. What is the cost impact of moving from on-premises to AWS? What is the cost impact of Lambda versus using EC2? Why, if the caching and content delivery network is working as expected, are 2,000 lamdbas a second required?

We have asked the BBC for more details. ®

Updated at 16:02 UTC on 5 November 2020 to add

The BBC's Matthew Clark got in touch to say: "Although EC2 Lambda compute cost is higher, the amount you need is less, offsetting this." He added, somewhat mysteriously since EC2 can autoscale, that: "With EC2, we provision web servers with plenty of capacity to handle sudden traffic changes (e.g. due to breaking news). Whereas with Lambda, we only pay for what we actually use."

To the question of why the org didn't use the opportunity of server-side rendering to deliver more human-readable HTML that would be better for parsing and accessibility tools, he responded: "The web page HTML looks different as it's largely generated by the framework used (React). The BBC has a very high bar for accessibility and performance, and we continue to test the site to ensure that it works well across browsers and screen readers." Lastly, we asked why, if the caching and content delivery network was working as expected, 2,000 Lamdbas a second were required.

Clark claimed: "The Lambdas are essential at handling updates so that the site remains up to date. Each BBC page typically involves multiple simple Lambda executions - the majority of which complete in under 150ms."

Send us news
96 Comments

This week in AI: Man arrested after cops say he rode in backseat of Autopilot Tesla

Plus: Non-profit ML groups snub sponsorship money from Google

In Brief Highway patrol officers in California arrested a man this week accused of riding in the backseat of his Tesla while it was under Autopilot.

The super-cruise-control software should have disengaged without him in the driver seat, yet it is claimed 25-year-old Param Sharma managed to bypass that requirement so that the vehicle would drive itself with him in the back. You're also supposed to have your hands on the wheel even while Autopilot is active so that you can take over from the computer system as necessary.

Following reports of a driverless Tesla Model 3, a highway patrol officer spotted the vehicle travelling east-bound towards the Bay Bridge in San Francisco, and attempted to stop it. It is alleged Sharma climbed back into the driver’s seat before he pulled over for the police.

Continue reading

China says its first Mars rover Zhurong has landed on the Red Planet

'An important step in our country’s interplanetary exploration journey' – state media

Updated China's Zhurong rover today touched down on Mars from the Tianwen-1 orbiter, the nation's state media says.

We're told the machine will take carry out self-tests, and try to move itself to explore the Red Planet's surface.

"On May 15, our country’s first Mars exploration mission, Tianwen-1, landed in a pre-selected landing zone in the southern Utopia Planitia of Mars, leaving a Chinese footprint on Mars for the first time. It marks an important step in our country’s interplanetary exploration journey," Xinhua reported at 0837 in Beijing (1737 PT, 0037 UTC).

Continue reading

Google leads Big Tech effort to ensure H-1B spouses can continue working in America

Coalition of 41 organizations oppose labor rule challenge

Google is spearheading an effort to save a visa rule that allows the spouses of H-1B visa holders awaiting green cards to work in the US.

On Friday, Google and 40 other companies and organizations filed an amicus brief supporting the Department of Homeland Security's (DHS) H-4 employment authorization document (H-4 EAD) program, which faces a legal challenge by a group called Save Jobs USA.

Save Jobs USA, an association representing Southern California Edison workers who claim they lost their jobs to H-1B visa holders, is suing DHS in a Washington, DC court to undo the rule.

Continue reading

AMD promises to spend $1.6bn on 12nm, 14nm chips from GlobalFoundries

Also wriggles out of exclusivity deal

Amid fears the global semiconductor crisis may last until 2023, AMD has opted to extend its purchase agreement with GlobalFoundries, giving it access to a greater proportion of the fabricator's output.

AMD disclosed the existence of the deal in an 8-K regulatory filing submitted to the SEC earlier this week. The company has committed to buy $1.6bn worth of 12nm and 14nm node silicon wafers between now and December 31, 2024. It did not disclose a breakdown of the costs nor the exact quantity of output it had secured.

Should AMD fail to meet its purchase obligation, it has committed to pay GlobalFoundries a portion of the difference between its planned and actual spend. AMD has also agreed to pre-pay for an unspecified portion of these wafers in advance.

Continue reading

Audacity's new management hits rewind on telemetry plans following community outrage

Sorry for trying to add it or sorry for cocking up the comms?

Amid the smell of burning rubber, the new managers of open-source audio editor Audacity have announced a U-turn on plans to introduce "basic telemetry" into the product.

Audacity pitched up under the umbrella of Muse Group earlier this month and professed itself to be both "scared and excited."

Mere days later, an impressive number of users went for the former option and expressed alarm at a GitHub request introducing "basic telemetry."

Continue reading

Apple's expert witness grilled by Epic over 'frictionless' spending outside the app

How easy would it be for customers to depart the walled garden, legal eagles ask economist

Epic Games' lawyers had a chance to put Apple's expert witness through the wringer in the latest from its California bench trial.

Counsel for Apple called to the stand Lorin Hitt, an academic from the prestigious Wharton Business School in Pennsylvania.

Hitt – who had been selected as expert witness for Apple – questioned whether iOS was as effective at locking in users as previously claimed, citing a 26 per cent switch rate. He also debated whether users remained loyal to a platform because of switching costs, or because they simply like it.

Continue reading

Facebook Giphy merger stays on ice after failed challenge to UK competition regulator

Problem was of social network's own making, says unimpressed judge

Facebook has failed to neutralise an order from Britain's competition regulator freezing its buyout of Giphy after having "sat on its hands" and failed to answer questions, the Court of Appeal has found.

Judge Sir Geoffrey Vos said "the central problem in this case was entirely of Facebook's own making" as he dismissed its attempt to overturn an Initial Enforcement Order (IEO) made by Britain's Competition and Markets Authority (CMA) last year.

That IEO blocked the Mark Zuckerberg-owned social network from finishing off its $400m buyout of Giphy, a supplier of web tracking beacons cunningly disguised as funny little animated images used to spice up online chats and comment sections.

Continue reading

10.8 million UK homes now have access to gigabit-capable broadband, with much of the legwork done by Virgin Media

That's 37% of the country covered, and BT is expected to pick up the pace too

A new Ofcom report shows the number of UK homes with access to gigabit-capable broadband hit 10.8 million in January, representing 37 per cent of households.

The figures were part of Ofcom's Interim Connected Nations report [PDF] and covered September 2020 to January 2021.

Overall, the number of gigabit-capable lines increased by 37 per cent against August's figure [PDF] of 7.9 million.

Continue reading

Tor users, beware: 'Scheme flooding' technique may be used to deanonymize you

By probing for installed apps with custom URL schemes, it's possible to build a 32-bit unique fingerprint

FingerprintJS, maker of a browser-fingerprinting library for fraud prevention, on Thursday said it has identified a more dubious fingerprinting technique capable of generating a consistent identifier across different desktop browsers, including the Tor Browser.

That means, for example, if you browse the web using Safari, Firefox, or Chrome for some websites, and use the Tor browser to anonymously view others, there is a possibility someone could link your browser histories across all those sessions using a unique identifier, potentially deanonymize you, and track you around the web.

Doing this is non-trivial, it can be very inaccurate or unreliable, and so this is more of a heads up than anything else.

Continue reading

NASA pops old-school worm logo onto Orion spacecraft

Will be visible from the launchpad ... when it finally gets there

NASA has slapped its worm logo on the side of the Crew Module Adaptor (CMA) for the Orion spacecraft as the first Artemis mission to the Moon inches closer.

The logo had already been stuck on the underside of the CMA last year, but sticking it on the side will ensure it is visible once the Orion spacecraft and its European-built service module are stacked atop the Space Launch System (SLS) rocket and wheeled out to Kennedy's pad 39B.

Continue reading

Hospitals cancel outpatient appointments as Irish health service struck by ransomware

Russia-based criminals pick soft target in hope of easy gains

Ireland's nationalised health service has shut down its IT systems following a "human-operated" Conti ransomware attack, causing a Dublin hospital to cancel outpatient appointments.

The country's Health Service Executive closed its systems down as a precaution, local reports from the Irish public service broadcaster RTÉ said, reporting that Dublin's Rotunda Hospital had cancelled appointments for outpatients – including many for pregnant women.

"The maternity hospital said all outpatient visits are cancelled - unless expectant mothers are 36 weeks pregnant or later," reported RTÉ, adding: "All gynaecology clinics are also cancelled today."

Continue reading