Netflix: Not interested in cloud 'heavy lifting'

Amazon gets them up in the sky, we bring the rain

If you're thinking that you might be able to sell Netflix an alternative cloud to Amazon Web Services, or better still, convince the online TV show and movie distributor to go back to building its own infrastructure using an open source cloud stack such as OpenStack in the hopes of saving money, just forget it.

Adrian Cockcroft, the cloud architect at Netflix, pours a whole lot of cold water on the OpenStack effort despite the fact that he very much wants Netflix to have an alternative to AWS.

Virtualization juggernaut VMware would like to convince IT shops that virtualization is the new hardware, and therefore worthy of the kinds of profit margins hardware used to command but no longer does (unless you are Apple). But if the way the top techies at Netflix are thinking is any indication, then the control point is a little higher up the stack somewhere between the infrastructure and platform cloud layers. And perhaps more significantly for new companies and the young entrepreneurs that will by and large be establishing them, they don't want to know about the underlying hardware that runs applications and will never consider directly investing in it.

Having been there already, explains Cockcroft on his personal blog, Netflix does not want to try to compete with Amazon in the cloud business – even if Amazon Prime is taking on Netflix in the online entertainment distribution business. (Netflix is one of the dominant players, but has not yet achieved verb status, as Google has in search.)

"I come to use clouds," explains Cockcroft, "because I work for a developer oriented company that has decided that building and running infrastructure on a global scale is undifferentiated heavy lifting, and we can leverage outside investment from AWS and others to do a better job than we could do ourselves, while we focus on the real job of developing global streaming to TVs."

And iPads and PCs. Don't forget those. In my house, we watch more Netflix on the iPad than we do on the widescreen.

Cockcroft joined Sun Microsystems back in 1988 and rose up through the ranks, helping the upstart Unix server vendor crack the corporate market and tune their Java applications until he moved to online auctioneer eBay in 2004 as an operations architect. Cockcroft was among those who set up eBay Labs and its public Web API stack. Three years later, Cockcroft jumped to Netflix, and now that company has become one of the biggest users – and biggest fans – of Amazon's cloudy infrastructure.

Neither Cockcroft nor Netflix has anything against OpenStack and the companies that are trying to create a more scalable alternative to the various cloud fabrics that are out there either trying to emulate AWS or are built using VMware's ESXi hypervisor and vCloud extensions or Microsoft's Hyper-V and soon Azure platform services. Cockcroft doesn't talk about VMware or Microsoft clouds in his blog, but it is pretty clear from his statements that he doesn't think OpenStack has much of a chance over the next couple of years of competing head-to-head with AWS.

"Some of the proponents of OpenStack argue that because it's an open source community project it will win in the end," writes Cockcroft. "I disagree. The most successful open source projects I can think of have a strong individual leader who spends a lot of time saying no to keep the project on track. Some of the least successful are large multi-vendor industry consortiums."

Cockcroft said that he attended an OpenStack development meeting a few weeks ago and was dismayed that OpenStack would not have a "production-level" release until the "Essex" version in March 2012. More importantly the OpenStackers were "the consortium of people who can't figure out how to compete with AWS on their own," and they were still down in the guts of the cloud fabric defining features by committee while AWS was adding features driven by real customers.

Brooks' Law

"While it seems obvious that adding more members to OpenStack is a good thing, in practice it will slow the project down," says Cockcroft. "I was once told that the way to kill a standards body or consortium is to keep inviting new people to join and adding to its scope. With the huge diversity of datacenter hardware and many vendors with a lot to lose if they get sidelined I expect OpenStack to fracture into multiple vendor specific 'stacks' with narrow test matrixes and extended features that lock customers in and don't interoperate well."

But don't get the wrong idea. Netflix is still rooting for OpenStack, and would probably like to see the full AWS API set implemented in OpenStack. This will almost certainly not happen unless the industry demands it over the heads of NASA and Rackspace Hosting, which founded the OpenStack cloud fabric project back in July 2010 after NASA became frustrated with the EC2-alike cloud created by Eucalyptus Systems.

"I haven't yet seen a viable alternative to AWS, but that doesn't mean I don't want to see one," explains Cockcroft. "My guess is that in about two to three years from now there may be a credible alternative. Netflix has already spent a lot of time helping AWS scale as we figured out our architecture, we don't want to do that again, so I'm also waiting for someone else (another large end-user) to kick the tires and prove that an alternative works."

Unlike the content delivery networks, clouds are not yet interoperable and not quite commodities. But in the end, the clouds that run Netflix's applications might be as anonymous and uninteresting as the CDNs that Netflix actually uses to stream those movies that its applications control. ®

Broader topics

Other stories you might like

  • Florida's content-moderation law kept on ice, likely unconstitutional, court says
    So cool you're into free speech because that includes taking down misinformation

    While the US Supreme Court considers an emergency petition to reinstate a preliminary injunction against Texas' social media law HB 20, the US Eleventh Circuit Court of Appeals on Monday partially upheld a similar injunction against Florida's social media law, SB 7072.

    Both Florida and Texas last year passed laws that impose content moderation restrictions, editorial disclosure obligations, and user-data access requirements on large online social networks. The Republican governors of both states justified the laws by claiming that social media sites have been trying to censor conservative voices, an allegation that has not been supported by evidence.

    Multiple studies addressing this issue say right-wing folk aren't being censored. They have found that social media sites try to take down or block misinformation, which researchers say is more common from right-leaning sources.

    Continue reading
  • US-APAC trade deal leaves out Taiwan, military defense not ruled out
    All fun and games until the chip factories are in the crosshairs

    US President Joe Biden has heralded an Indo-Pacific trade deal signed by several nations that do not include Taiwan. At the same time, Biden warned China that America would help defend Taiwan from attack; it is home to a critical slice of the global chip industry, after all. 

    The agreement, known as the Indo-Pacific Economic Framework (IPEF), is still in its infancy, with today's announcement enabling the United States and the other 12 participating countries to begin negotiating "rules of the road that ensure [US businesses] can compete in the Indo-Pacific," the White House said. 

    Along with America, other IPEF signatories are Australia, Brunei, India, Indonesia, Japan, South Korea, Malaysia, New Zealand, the Philippines, Singapore, Thailand and Vietnam. Combined, the White House said, the 13 countries participating in the IPEF make up 40 percent of the global economy. 

    Continue reading
  • 381,000-plus Kubernetes API servers 'exposed to internet'
    Firewall isn't a made-up word from the Hackers movie, people

    A large number of servers running the Kubernetes API have been left exposed to the internet, which is not great: they're potentially vulnerable to abuse.

    Nonprofit security organization The Shadowserver Foundation recently scanned 454,729 systems hosting the popular open-source platform for managing and orchestrating containers, finding that more than 381,645 – or about 84 percent – are accessible via the internet to varying degrees thus providing a cracked door into a corporate network.

    "While this does not mean that these instances are fully open or vulnerable to an attack, it is likely that this level of access was not intended and these instances are an unnecessarily exposed attack surface," Shadowserver's team stressed in a write-up. "They also allow for information leakage on version and build."

    Continue reading
  • A peek into Gigabyte's GPU Arm for AI, HPC shops
    High-performance platform choices are going beyond the ubiquitous x86 standard

    Arm-based servers continue to gain momentum with Gigabyte Technology introducing a system based on Ampere's Altra processors paired with Nvidia A100 GPUs, aimed at demanding workloads such as AI training and high-performance compute (HPC) applications.

    The G492-PD0 runs either an Ampere Altra or Altra Max processor, the latter delivering 128 64-bit cores that are compatible with the Armv8.2 architecture.

    It supports 16 DDR4 DIMM slots, which would be enough space for up to 4TB of memory if all slots were filled with 256GB memory modules. The chassis also has space for no fewer than eight Nvidia A100 GPUs, which would make for a costly but very powerful system for those workloads that benefit from GPU acceleration.

    Continue reading
  • GitLab version 15 goes big on visibility and observability
    GitOps fans can take a spin on the free tier for pull-based deployment

    One-stop DevOps shop GitLab has announced version 15 of its platform, hot on the heels of pull-based GitOps turning up on the platform's free tier.

    Version 15.0 marks the arrival of GitLab's next major iteration and attention this time around has turned to visibility and observability – hardly surprising considering the acquisition of OpsTrace as 2021 drew to a close, as well as workflow automation, security and compliance.

    GitLab puts out monthly releases –  hitting 15.1 on June 22 –  and we spoke to the company's senior director of Product, Kenny Johnston, at the recent Kubecon EU event, about what will be added to version 15 as time goes by. During a chat with the company's senior director of Product, Kenny Johnston, at the recent Kubecon EU event, The Register was told that this was more where dollars were being invested into the product.

    Continue reading
  • To multicloud, or not: Former PayPal head of engineering weighs in
    Not everyone needs it, but those who do need to consider 3 things, says Asim Razzaq

    The push is on to get every enterprise thinking they're missing out on the next big thing if they don't adopt a multicloud strategy.

    That shove in the multicloud direction appears to be working. More than 75 percent of businesses are now using multiple cloud providers, according to Gartner. That includes some big companies, like Boeing, which recently chose to spread its bets across AWS, Google Cloud and Azure as it continues to eliminate old legacy systems. 

    There are plenty of reasons to choose to go with multiple cloud providers, but Asim Razzaq, CEO and founder at cloud cost management company Yotascale, told The Register that choosing whether or not to invest in a multicloud architecture all comes down to three things: How many different compute needs a business has, budget, and the need for redundancy. 

    Continue reading

Biting the hand that feeds IT © 1998–2022