Social networks talk hidden architectures

Back-stage bytes


Social networks are almost pervasive. Even if you're not actually on one, it's becoming impossible to avoid hearing of them and often it's the same networks that keep popping up, such as Facebook or MySpace.

While they might be well known, though, the companies tend not to discuss the architectures that underpin their services. Are they running some cleaver RESTful science behind the scenes or are they just using a vanilla combo of PHP on lots and lots of Windows or Linux servers?

Facebook, MySpace, Digg and Ning recently shared their trials and tribulations at the QCon conference in San Francisco, California.

Dan Farino, chief systems architect at MySpace.com, said his site started with a very small architecture and scaled out. He focused on monitoring and administration on a Windows network and the challenge of keeping the system running on thousands of servers.

"Yes, we run Windows!" he said. "It's actually a pretty good server platform. IIS is a pretty good web server. Tuned properly, it's going to serve pages; it's not going to crash or tip over when it gets Slashdotted with two requests. It's pretty solid. What isn't solid about Windows is the large-scale management tools."

MySpace relies on 4,500-plus Windows-based web servers. A middle-tier cache has been added, but it's still "basically a bunch of servers from an operational perspective," Farino said.

Data challenge

A key challenge for MySpace was to come up with the tools for quickly collecting data when there's a problem, so that those problems could be analyzed and avoided in the future. To collect the operational data needed for proper analysis, Farino developed a custom performance monitoring system that tracks real-time CPU requests queued, request per second, and similar information live across the company's server farm.

Digg.com is the largest content aggregator on the Net, with 3.5 million registered users. Lead architect Joe Stump gave a peak behind the scenes of a system that handles about 15,000 requests per second and reports serving approximately 26 million unique visitors per month.

Stump described the system's innards as "an architecture in transition."

"I call it that because Digg started out as a harebrained idea," Stump said. "It wasn't one of those projects where you start out saying, what happens if in a year I'm doing 300 thousand diggs a day and 15 billion common diggs?"

Digg.com uses MySQL, but on top of that is a library designed to interact with the DBMS in a specialized manner, Stump said. "Scaling is specialization," he added. "You can't just take a commercial product off the shelf, throw it into production and hope it works. The way you normally scale things is to take a few different components, layer something on top of it that has your specialization stuff in it."

Facebook has evolved into a huge social network. It has more than 120 million active users and 10 billion photos, and serves up 50 billion page views per month.

Similar topics


Other stories you might like

  • Prisons transcribe private phone calls with inmates using speech-to-text AI

    Plus: A drug designed by machine learning algorithms to treat liver disease reaches human clinical trials and more

    In brief Prisons around the US are installing AI speech-to-text models to automatically transcribe conversations with inmates during their phone calls.

    A series of contracts and emails from eight different states revealed how Verus, an AI application developed by LEO Technologies and based on a speech-to-text system offered by Amazon, was used to eavesdrop on prisoners’ phone calls.

    In a sales pitch, LEO’s CEO James Sexton told officials working for a jail in Cook County, Illinois, that one of its customers in Calhoun County, Alabama, uses the software to protect prisons from getting sued, according to an investigation by the Thomson Reuters Foundation.

    Continue reading
  • Battlefield 2042: Please don't be the death knell of the franchise, please don't be the death knell of the franchise

    Another terrible launch, but DICE is already working on improvements

    The RPG Greetings, traveller, and welcome back to The Register Plays Games, our monthly gaming column. Since the last edition on New World, we hit level cap and the "endgame". Around this time, item duping exploits became rife and every attempt Amazon Games made to fix it just broke something else. The post-level 60 "watermark" system for gear drops is also infuriating and tedious, but not something we were able to address in the column. So bear these things in mind if you were ever tempted. On that note, it's time to look at another newly released shit show – Battlefield 2042.

    I wanted to love Battlefield 2042, I really did. After the bum note of the first-person shooter (FPS) franchise's return to Second World War theatres with Battlefield V (2018), I stupidly assumed the next entry from EA-owned Swedish developer DICE would be a return to form. I was wrong.

    The multiplayer military FPS market is dominated by two forces: Activision's Call of Duty (COD) series and EA's Battlefield. Fans of each franchise are loyal to the point of zealotry with little crossover between player bases. Here's where I stand: COD jumped the shark with Modern Warfare 2 in 2009. It's flip-flopped from WW2 to present-day combat and back again, tried sci-fi, and even the Battle Royale trend with the free-to-play Call of Duty: Warzone (2020), which has been thoroughly ruined by hackers and developer inaction.

    Continue reading
  • American diplomats' iPhones reportedly compromised by NSO Group intrusion software

    Reuters claims nine State Department employees outside the US had their devices hacked

    The Apple iPhones of at least nine US State Department officials were compromised by an unidentified entity using NSO Group's Pegasus spyware, according to a report published Friday by Reuters.

    NSO Group in an email to The Register said it has blocked an unnamed customers' access to its system upon receiving an inquiry about the incident but has yet to confirm whether its software was involved.

    "Once the inquiry was received, and before any investigation under our compliance policy, we have decided to immediately terminate relevant customers’ access to the system, due to the severity of the allegations," an NSO spokesperson told The Register in an email. "To this point, we haven’t received any information nor the phone numbers, nor any indication that NSO’s tools were used in this case."

    Continue reading

Biting the hand that feeds IT © 1998–2021