Software

Devops

Happy birthday, Python, you're 30 years old this week: Easy to learn, and the right tool at the right time

Popular programming language, at the top of its game, still struggles to please everyone


Feature The 30th anniversary of Python this week finds the programming language at the top of its game, but not without challenges.

"I do believe that Python just doesn’t have the right priorities these days," said Armin Ronacher, director of engineering at software monitoring biz Sentry and creator of Flask, the popular Python web app framework, in an email interview with The Register.

Ronacher, a prolific Python contributor, remains a fan of the language. He credits Python's success to being both easy to learn and having an implementation that was easy to hack. And in its early years, Python didn't have a lot of competitors with those same characteristics, he said.

The hackability of the language enabled many of the projects that made it successful

"The hackability of the language enabled many of the projects that made it successful, such as NumPy and others, which extended the language through extension modules written in C that would have been hard to do with Python alone at the time," he said. "Some of the functionality to enable libraries like NumPy were added right to the language itself to enable these more advanced use cases."

Ronacher expressed appreciation for Python's readability, at least initially.

"It’s easy to read, it wasn’t an overly complex language for a long time, and it gives you a lot of access to the internals," he said. "The latter allows you to introspect the runtime without many penalties, which in turn means that it’s an interesting language to build web services on top. When something goes wrong in production you can easily figure out what was happening."

He also pointed to the relatively simple runtime, which makes runtime performance more predictable. "While it’s not a very fast language, it compensates for this somewhat because the reference counting semantics often mean that the memory usage is somewhat predictable in production environments," he said.

At the same time, Ronacher takes issue with the path Python has taken recently.

"Over the last few years, Python hasn't made the most amazing decisions," he said. "For instance, I’m not a fan of how Unicode was approached. With Python 3, I wish a Unicode model more like Rust would have been approached that just declares strings to be UTF-8 in memory. Python 3 is very wasteful with memory when it comes to Unicode to permit direct indexing into characters, which is not that useful anyways with modern Unicode."

He also took issue with the focus of Python's core developers.

"Many features are landing that are making the language much more complex to learn, such as the async IO system, the way the typing support works, and the new match statement," he said. "Meanwhile, essential features such as a better packaging story are still absent."

Essential features such as a better packaging story are still absent

The shortcomings of Python's software packaging tools – the software used to set up Python environments and to download, install, and manage libraries – have been an issue for years. It was bad enough that cartoonist Randall Munroe, on April 30, 2018, penned an xkcd comic on the subject.

Things have improved somewhat since then. In 2019, the Python Software Foundation awarded the Packaging Working Group $407,000, courtesy of Mozilla and the Chan Zuckerberg Initiative, to renovate the pip package management tool in 2020.

Nevertheless, Ronacher said he hopes Python's core developers focus on improving packaging and on adding the ability to load different versions of the same library side by side.

"It’s much more complex to install packages in the Python ecosystem than others and the packaging infrastructure is too disconnected from the core language development," he said.

"Whereas Node now comes with npm out of the box and Rust develops the cargo package manager alongside the language, Python still does not consider packaging to be part of language development. As a result, there are countless competing efforts that are all pieced together."

"Whereas a Rust programmer can just download the language and use the integrated rustup+cargo tools for everything, Python programmers need to juggle many different tools to accomplish something similar but those tools are not developed in unison," Ronacher explained. "Unlike all other modern languages, Python also can only load one version of a dependency. This means that your entire software project needs to agree on a compatible version, which becomes harder the larger the ecosystem grows and the faster it moves."

Even though Python's packaging story still suffers by comparison to Rust's well-regarded Cargo system, the language has never been more popular. Its maintainers must be doing something right.

Who's in control?

Named as a nod to British comedy troupe Monty Python, the language has become the second or third most popular choice, depending on who you ask, for those writing computer code.

The language's creator, Guido van Rossum, relinquished his role overseeing the language and his honorary epithet, Benevolent Dictator for Life, in July 2018. This was after a fractious debate over the addition of a new language feature, the "Walrus operator" (PEP 572), that left van Rossum frustrated by online animosity.

The Register asked van Rossum to comment for the occasion but he declined, stating that he's not all that interested in promoting himself.

Python is currently managed by the Python Steering Council, which consists of five people who serve for the duration of a feature release, the most recent of which was Python 3.9.0 last October. Currently, the group includes: Barry Warsaw, Brett Cannon, Carol Willing, Pablo Galindo Salgado, and Thomas Wouters.

These five oversee technical changes to Python and manage the community governance process – a process based on Python Enhancement Proposals (PEPs). They coordinate the contributions of more than 90 active core developers and other members of the Python community.

Python swallows Java to become second-most popular programming language... according to this index

READ MORE

For the past 20 years, as of March 6, 2021, the Python Software Foundation (PSF) has supported the language's development. Its aim is to "promote, protect, and advance the Python programming language, and to support and facilitate the growth of a diverse and international community of Python programmers."

"The PSF has a little bit been distanced from the actual maintenance of the language itself," said Ewa Jodlowska, executive director of the PSF, in an interview with The Register. "But through the creation of the Steering Council, that has created a vessel for us to be able to communicate with and work on future funding requests that might aid them in some things that they lack."

The Python Foundation has found it difficult to meet its funding goals during the COVID-19 pandemic. According to Jodlowska, much of the foundation's revenue has traditionally come from PyCon US. The event was virtual last year and will be again this year, limiting the potential income.

Jodlowska said one of the PSF's goals this year is to hire a full-time core developer, an ambition fulfilled on February 11, when Google announced plans to donate more than $350,000 to the PSF to support three projects: a malware detection system for the Python Package Index (PyPI); improvements in Python tools and services; and paying for a CPython Developer-in-Residence for 2021, to work on language maintenance.

CPython is the reference implementation of the language, written in C, the one you download from Python.org. But there are others like IronPython (C#), Jython (Java), and PyPy (RPython, a Python subset).

Asked what has made Python so successful, Jodlowska cited the importance of the Python community and the role its Code of Conduct plays as a support structure.

"Diversity for the maintainers and core developers of Python is being addressed in several ways," Jodlowska explained.

The Code of Conduct, she said, is now being enforced by the Steering Council. That's a big deal, she said, "because prior to the Steering Council existing there really was no enforcement for keeping community discussions civil and welcoming."

In theory, the Steering Council's Code of Conduct enforcement will prevent situations like the Walrus operator debate that drove van Rossum to give up his governance role. But to judge by remarks from Python Steering Council member Brett Cannon, moderating community debate simply moves the point of friction from the community to its leadership.

We are still very much a volunteer-run project, but our size demands a lot of time to keep running

Asked via email about the greatest challenge facing the Python community, Cannon said it's just trying to keep up with the size and volume of the project.

"We are still very much a volunteer-run project, but our size demands a lot of time to keep running," he said. "Tack on the usual naysayers for any decision made and it's a lot to manage both from a time and emotion standpoint."

At companies like Facebook and Microsoft that moderate the worst sorts of content, the workers who screen toxic posts and violent videos need psychological support. Managing opinionated developers in the Python community may not generate comparable levels of stress, but keeping things civil still appears to take a mental toll, particularly among volunteers.

For the Python community, bringing new people in so others can step back or delegate may help mitigate that sense of siege. Jodlowska credits efforts by Python core developers to keep the community vital. "A lot of the current core developers, for example, on their own time mentor others who are interested in becoming core developers," she said. "And there's definitely a steady stream of new incomers that way."

Blessing and a curse

Many of the current leading programming languages have strong corporate associations. Java is the child of Sun and later Oracle. Swift is mainly an Apple affair. C# is tied to Microsoft. Go and Dart arose from Google. JavaScript escaped from Netscape, which provided the browser source code that led to Mozilla, the incubator of Rust.

But Python has never enjoyed a doting or overprotective corporate parent, which according to Cannon has been both a liability and a benefit.

Apple, Microsoft, PayPal among 35 organizations compromised by evil twin dependencies attack

READ MORE

"It has hurt us as we have to go out to go searching for funding for everything and we lack paid developers to help keep things running (the best estimate we have is there's a cumulative total of about three to four devs putting in paid time on Python, and one of those is a single person with about 80 per cent time; rest is a smattering from several folks)," he said. "But being independent also means our users never have to worry about us being directed by business needs and can instead always focus on our users and their needs (with the limited resources we have). So it has pluses and minuses."

Asked about how Python goes about reconciling interest in new features with concerns about complexity, Cannon acknowledged that it's a constant challenge.

"There's always tension between expanding the language to make developers even more productive while letting it continue to 'fit your brain,'" said Cannon.

He said it's a balance of how early on would a new user likely come across a feature; the ease with which someone would recognize, if not comprehend, a feature when seen for the first time; the difficulty of searching for answer to find out how a feature works; and the extent to which a feature's function is memorable to those trying to learn it.

Python's other noteworthy obstacle is its scarcity on mobile devices. There are ways to run Python code on phones, like the Kivy framework, but Python isn't the first choice of most mobile app developers.

"I hope it will improve in the next three to five years," said Cannon. "There are several groups that are actively trying to tackle the problem from different angles, but they all require tackling big, hard problems."

Thirty years on, Python deserves recognition for what it has accomplished but it can't rest on its laurels. Rival programming languages like Julia and R in data science, and Go in cloud-native applications, have been turning heads. And the need for greater memory safety, to reduce security risks, has helped push TypeScript and Rust into the spotlight. Uneasy lies the head that wears a crown. ®

Send us news
78 Comments

China says its first Mars rover Zhurong has landed on the Red Planet

'An important step in our country’s interstellar exploration journey' – state media

Updated China's Zhurong rover today touched down on Mars from the Tianwen-1 orbiter, the nation's state media says.

We're told the machine will take carry out self-tests, and try to move itself to explore the Red Planet's surface.

"On May 15, our country’s first Mars exploration mission, Tianwen-1, landed in a pre-selected landing zone in the southern Utopia Planitia of Mars, leaving a Chinese footprint on Mars for the first time. It marks an important step in our country’s interstellar exploration journey," Xinhua reported at 0837 in Beijing (1737 PT, 0037 UTC).

Continue reading

Google leads Big Tech effort to ensure H-1B spouses can continue working in America

Coalition of 41 organizations oppose labor rule challenge

Google is spearheading an effort to save a visa rule that allows the spouses of H-1B visa holders awaiting green cards to work in the US.

On Friday, Google and 40 other companies and organizations filed an amicus brief supporting the Department of Homeland Security's (DHS) H-4 employment authorization document (H-4 EAD) program, which faces a legal challenge by a group called Save Jobs USA.

Save Jobs USA, an association representing Southern California Edison workers who claim they lost their jobs to H-1B visa holders, is suing DHS in a Washington, DC court to undo the rule.

Continue reading

AMD promises to spend $1.6bn on 12nm, 14nm chips from GlobalFoundries

Also wriggles out of exclusivity deal

Amid fears the global semiconductor crisis may last until 2023, AMD has opted to extend its purchase agreement with GlobalFoundries, giving it access to a greater proportion of the fabricator's output.

AMD disclosed the existence of the deal in an 8-K regulatory filing submitted to the SEC earlier this week. The company has committed to buy $1.6bn worth of 12nm and 14nm node silicon wafers between now and December 31, 2024. It did not disclose a breakdown of the costs nor the exact quantity of output it had secured.

Should AMD fail to meet its purchase obligation, it has committed to pay GlobalFoundries a portion of the difference between its planned and actual spend. AMD has also agreed to pre-pay for an unspecified portion of these wafers in advance.

Continue reading

Audacity's new management hits rewind on telemetry plans following community outrage

Sorry for trying to add it or sorry for cocking up the comms?

Amid the smell of burning rubber, the new managers of open-source audio editor Audacity have announced a U-turn on plans to introduce "basic telemetry" into the product.

Audacity pitched up under the umbrella of Muse Group earlier this month and professed itself to be both "scared and excited."

Mere days later, an impressive number of users went for the former option and expressed alarm at a GitHub request introducing "basic telemetry."

Continue reading

Apple's expert witness grilled by Epic over 'frictionless' spending outside the app

How easy would it be for customers to depart the walled garden, legal eagles ask economist

Epic Games' lawyers had a chance to put Apple's expert witness through the wringer in the latest from its California bench trial.

Counsel for Apple called to the stand Lorin Hitt, an academic from the prestigious Wharton Business School in Pennsylvania.

Hitt – who had been selected as expert witness for Apple – questioned whether iOS was as effective at locking in users as previously claimed, citing a 26 per cent switch rate. He also debated whether users remained loyal to a platform because of switching costs, or because they simply like it.

Continue reading

Facebook Giphy merger stays on ice after failed challenge to UK competition regulator

Problem was of social network's own making, says unimpressed judge

Facebook has failed to neutralise an order from Britain's competition regulator freezing its buyout of Giphy after having "sat on its hands" and failed to answer questions, the Court of Appeal has found.

Judge Sir Geoffrey Vos said "the central problem in this case was entirely of Facebook's own making" as he dismissed its attempt to overturn an Initial Enforcement Order (IEO) made by Britain's Competition and Markets Authority (CMA) last year.

That IEO blocked the Mark Zuckerberg-owned social network from finishing off its $400m buyout of Giphy, a supplier of web tracking beacons cunningly disguised as funny little animated images used to spice up online chats and comment sections.

Continue reading

10.8 million UK homes now have access to gigabit-capable broadband, with much of the legwork done by Virgin Media

That's 37% of the country covered, and BT is expected to pick up the pace too

A new Ofcom report shows the number of UK homes with access to gigabit-capable broadband hit 10.8 million in January, representing 37 per cent of households.

The figures were part of Ofcom's Interim Connected Nations report [PDF] and covered September 2020 to January 2021.

Overall, the number of gigabit-capable lines increased by 37 per cent against August's figure [PDF] of 7.9 million.

Continue reading

Tor users, beware: 'Scheme flooding' technique may be used to deanonymize you

By probing for installed apps with custom URL schemes, it's possible to build a 32-bit unique fingerprint

FingerprintJS, maker of a browser-fingerprinting library for fraud prevention, on Thursday said it has identified a more dubious fingerprinting technique capable of generating a consistent identifier across different desktop browsers, including the Tor Browser.

That means, for example, if you browse the web using Safari, Firefox, or Chrome for some websites, and use the Tor browser to anonymously view others, there is a possibility someone could link your browser histories across all those sessions using a unique identifier, potentially deanonymize you, and track you around the web.

Doing this is non-trivial, it can be very inaccurate or unreliable, and so this is more of a heads up than anything else.

Continue reading

NASA pops old-school worm logo onto Orion spacecraft

Will be visible from the launchpad ... when it finally gets there

NASA has slapped its worm logo on the side of the Crew Module Adaptor (CMA) for the Orion spacecraft as the first Artemis mission to the Moon inches closer.

The logo had already been stuck on the underside of the CMA last year, but sticking it on the side will ensure it is visible once the Orion spacecraft and its European-built service module are stacked atop the Space Launch System (SLS) rocket and wheeled out to Kennedy's pad 39B.

Continue reading

Hospitals cancel outpatient appointments as Irish health service struck by ransomware

Russia-based criminals pick soft target in hope of easy gains

Ireland's nationalised health service has shut down its IT systems following a "human-operated" Conti ransomware attack, causing a Dublin hospital to cancel outpatient appointments.

The country's Health Service Executive closed its systems down as a precaution, local reports from the Irish public service broadcaster RTÉ said, reporting that Dublin's Rotunda Hospital had cancelled appointments for outpatients – including many for pregnant women.

"The maternity hospital said all outpatient visits are cancelled - unless expectant mothers are 36 weeks pregnant or later," reported RTÉ, adding: "All gynaecology clinics are also cancelled today."

Continue reading

Rapping otters and automated database knob-twiddling: An obvious combination in some universe or other

OtterTune to compete with Oracle automation, but also for open source databases

A university spin-out startup has announced a private beta of an automated database tuning service which its founder claims can double the performance or halve the cost of the popular AWS Relational Database Service.

Among its marketing hype, though, is the, erm, novel approach of launching a hip-hop album of beats and screeching otters. More of that later.

Originating from a project at Carnegie Mellon Database Group, OtterTune is based on the idea you can use machine learning to identify the optimal setting for database parameter knobs, a task well beyond most developers and something even seasoned DBAs struggle with, given the number of databases on the market that they might be required to manage.

Continue reading