Amazon's 'schizophrenic' open source selfishness scares off potential talent, say insiders

Moles blame Bezos for paltry code sharing


Exclusive Amazon is one of the most technically influential companies operating today – but you wouldn't know it, thanks to a dearth of published research papers and negligible code contributions to the open-source projects it relies on.

This, according to multiple insiders, is becoming a problem. The corporation is described as a "black hole" because improvements and fixes for the open-source software it uses rarely see the light of day. And, we're told, that policy of secrecy comes right from the top – and it's driving talent into the arms of its rivals.

Over the past three months, The Register has interviewed numerous Amazon insiders* about the tech giant's attitude toward open-source software, and how the cash-flush business interacts with the tech community.

The impression built up is of a company that harvests code from vast fields of open-source software while obscuring its code donations and distancing itself from the wider world of computing.

"Amazon cannot exist without open source," said one former Amazonian.

"All of the standard web technologies were in there," said another ex-employee. "We had everything from Perl to Java to C++. In a number of cases where performance [was] really important, we looked at the [Linux] kernel ... We really ran the gamut of usage."

But you wouldn't know this from Amazon's public contributions to open-source efforts: as far as El Reg can tell, the internet titan has submitted patches and other improvements to very few projects. When it does contribute, it does so typically via a third party, usually an employee's personal account that is not explicitly linked to Amazon.

These code contributions include additions to Apache Hadoop, jQuery, the Linux kernel and Ruby, we understand.

"Within Amazon it was well known that [CEO] Jeff Bezos didn't think that Amazon would gain from participating in open source except in very limited ways at the fringes of its tech," said one ex-Amazonian. "Amazon really kept its code closed."

Of course, the web giant is under no obligation to share its enhancements for open-source-licensed software if it's not distributing the code beyond its walls (or if the license doesn't require it to in any case).

This secretiveness may give Amazon a competitive edge in the short term, but – crucially – there's evidence that it could be damaging the company in the long term because few of today's most talented technologists want to work at a company that shuts them off from the wider technical community.

Many companies choose not to donate code and instead involve themselves in the community in other ways, such as attending or organizing conferences and publishing academic papers, but here Amazon is withdrawn as well.

"When I arrived [at Amazon], one of the things that was very striking was very few people went to conferences except quiet observers," explained another. "We lost a number of key technical prospects in terms of hiring people."

Where employees from Google, Microsoft, Facebook, Oracle, IBM, and others regularly speak at conferences and meet-ups around the world, Amazon's employees are instead trained to fade into the background at conferences, revealing little about themselves or their employer.

Your El Reg correspondent saw this himself when he attended Symposium on Operating Systems Principles (SOSP), the influential computer-science confab, last year. That was a week spent walking the halls of the cavernous Nemacolin Woodlands Resort in Pennsylvania, brushing shoulders with luminaries from Google, Microsoft, Cambridge University, Carnegie Mellon, and others.

Though there were a large number of Amazon.com and Amazon Web Services developers there, none of them spoke up in group sessions or made explicit reference to their company. The only Amazon-related presentation your Reg writer saw was given by Peter Vosshall, a senior engineer at Amazon.com, and this was more about the culture and management of a technical organization than a disclosure of precise techniques.

This secretiveness, "comes from Jeff," claimed another source. "It's passed down in HR training and policy. It's all very clear."

Though a select few are permitted to give public talks, when they do, they disclose far less information about their company's technology than their peers. El Reg has been to numerous obscure presentations by Amazonians where the audience is left dissatisfied by the paucity of the knowledge shared.

"Amazon behaves a lot like a classified military agency," explained another ex-Amazonian.


Other stories you might like

  • Colocation consolidation: Analysts look at what's driving the feeding frenzy
    Sometimes a half-sized shipping container at the base of a cell tower is all you need

    Analysis Colocation facilities aren't just a place to drop a couple of servers anymore. Many are quickly becoming full-fledged infrastructure-as-a-service providers as they embrace new consumption-based models and place a stronger emphasis on networking and edge connectivity.

    But supporting the growing menagerie of value-added services takes a substantial footprint and an even larger customer base, a dynamic that's driven a wave of consolidation throughout the industry, analysts from Forrester Research and Gartner told The Register.

    "You can only provide those value-added services if you're big enough," Forrester research director Glenn O'Donnell said.

    Continue reading
  • D-Wave deploys first US-based Advantage quantum system
    For those that want to keep their data in the homeland

    Quantum computing outfit D-Wave Systems has announced availability of an Advantage quantum computer accessible via the cloud but physically located in the US, a key move for selling quantum services to American customers.

    D-Wave reported that the newly deployed system is the first of its Advantage line of quantum computers available via its Leap quantum cloud service that is physically located in the US, rather than operating out of D-Wave’s facilities in British Columbia.

    The new system is based at the University of Southern California, as part of the USC-Lockheed Martin Quantum Computing Center hosted at USC’s Information Sciences Institute, a factor that may encourage US organizations interested in evaluating quantum computing that are likely to want the assurance of accessing facilities based in the same country.

    Continue reading
  • Bosses using AI to hire candidates risk discriminating against disabled applicants
    US publishes technical guide to help organizations avoid violating Americans with Disabilities Act

    The Biden administration and Department of Justice have warned employers using AI software for recruitment purposes to take extra steps to support disabled job applicants or they risk violating the Americans with Disabilities Act (ADA).

    Under the ADA, employers must provide adequate accommodations to all qualified disabled job seekers so they can fairly take part in the application process. But the increasing rollout of machine learning algorithms by companies in their hiring processes opens new possibilities that can disadvantage candidates with disabilities. 

    The Equal Employment Opportunity Commission (EEOC) and the DoJ published a new document this week, providing technical guidance to ensure companies don't violate ADA when using AI technology for recruitment purposes.

    Continue reading
  • How ICE became a $2.8b domestic surveillance agency
    Your US tax dollars at work

    The US Immigration and Customs Enforcement (ICE) agency has spent about $2.8 billion over the past 14 years on a massive surveillance "dragnet" that uses big data and facial-recognition technology to secretly spy on most Americans, according to a report from Georgetown Law's Center on Privacy and Technology.

    The research took two years and included "hundreds" of Freedom of Information Act requests, along with reviews of ICE's contracting and procurement records. It details how ICE surveillance spending jumped from about $71 million annually in 2008 to about $388 million per year as of 2021. The network it has purchased with this $2.8 billion means that "ICE now operates as a domestic surveillance agency" and its methods cross "legal and ethical lines," the report concludes.

    ICE did not respond to The Register's request for comment.

    Continue reading
  • Fully automated AI networks less than 5 years away, reckons Juniper CEO
    You robot kids, get off my LAN

    AI will completely automate the network within five years, Juniper CEO Rami Rahim boasted during the company’s Global Summit this week.

    “I truly believe that just as there is this need today for a self-driving automobile, the future is around a self-driving network where humans literally have to do nothing,” he said. “It's probably weird for people to hear the CEO of a networking company say that… but that's exactly what we should be wishing for.”

    Rahim believes AI-driven automation is the latest phase in computer networking’s evolution, which began with the rise of TCP/IP and the internet, was accelerated by faster and more efficient silicon, and then made manageable by advances in software.

    Continue reading

Biting the hand that feeds IT © 1998–2022