Software

Devops

Happy birthday, Python, you're 30 years old this week: Easy to learn, and the right tool at the right time

Popular programming language, at the top of its game, still struggles to please everyone


Feature The 30th anniversary of Python this week finds the programming language at the top of its game, but not without challenges.

"I do believe that Python just doesn’t have the right priorities these days," said Armin Ronacher, director of engineering at software monitoring biz Sentry and creator of Flask, the popular Python web app framework, in an email interview with The Register.

Ronacher, a prolific Python contributor, remains a fan of the language. He credits Python's success to being both easy to learn and having an implementation that was easy to hack. And in its early years, Python didn't have a lot of competitors with those same characteristics, he said.

The hackability of the language enabled many of the projects that made it successful

"The hackability of the language enabled many of the projects that made it successful, such as NumPy and others, which extended the language through extension modules written in C that would have been hard to do with Python alone at the time," he said. "Some of the functionality to enable libraries like NumPy were added right to the language itself to enable these more advanced use cases."

Ronacher expressed appreciation for Python's readability, at least initially.

"It’s easy to read, it wasn’t an overly complex language for a long time, and it gives you a lot of access to the internals," he said. "The latter allows you to introspect the runtime without many penalties, which in turn means that it’s an interesting language to build web services on top. When something goes wrong in production you can easily figure out what was happening."

He also pointed to the relatively simple runtime, which makes runtime performance more predictable. "While it’s not a very fast language, it compensates for this somewhat because the reference counting semantics often mean that the memory usage is somewhat predictable in production environments," he said.

At the same time, Ronacher takes issue with the path Python has taken recently.

"Over the last few years, Python hasn't made the most amazing decisions," he said. "For instance, I’m not a fan of how Unicode was approached. With Python 3, I wish a Unicode model more like Rust would have been approached that just declares strings to be UTF-8 in memory. Python 3 is very wasteful with memory when it comes to Unicode to permit direct indexing into characters, which is not that useful anyways with modern Unicode."

He also took issue with the focus of Python's core developers.

"Many features are landing that are making the language much more complex to learn, such as the async IO system, the way the typing support works, and the new match statement," he said. "Meanwhile, essential features such as a better packaging story are still absent."

Essential features such as a better packaging story are still absent

The shortcomings of Python's software packaging tools – the software used to set up Python environments and to download, install, and manage libraries – have been an issue for years. It was bad enough that cartoonist Randall Munroe, on April 30, 2018, penned an xkcd comic on the subject.

Things have improved somewhat since then. In 2019, the Python Software Foundation awarded the Packaging Working Group $407,000, courtesy of Mozilla and the Chan Zuckerberg Initiative, to renovate the pip package management tool in 2020.

Nevertheless, Ronacher said he hopes Python's core developers focus on improving packaging and on adding the ability to load different versions of the same library side by side.

"It’s much more complex to install packages in the Python ecosystem than others and the packaging infrastructure is too disconnected from the core language development," he said.

"Whereas Node now comes with npm out of the box and Rust develops the cargo package manager alongside the language, Python still does not consider packaging to be part of language development. As a result, there are countless competing efforts that are all pieced together."

"Whereas a Rust programmer can just download the language and use the integrated rustup+cargo tools for everything, Python programmers need to juggle many different tools to accomplish something similar but those tools are not developed in unison," Ronacher explained. "Unlike all other modern languages, Python also can only load one version of a dependency. This means that your entire software project needs to agree on a compatible version, which becomes harder the larger the ecosystem grows and the faster it moves."

Even though Python's packaging story still suffers by comparison to Rust's well-regarded Cargo system, the language has never been more popular. Its maintainers must be doing something right.

Who's in control?

Named as a nod to British comedy troupe Monty Python, the language has become the second or third most popular choice, depending on who you ask, for those writing computer code.

The language's creator, Guido van Rossum, relinquished his role overseeing the language and his honorary epithet, Benevolent Dictator for Life, in July 2018. This was after a fractious debate over the addition of a new language feature, the "Walrus operator" (PEP 572), that left van Rossum frustrated by online animosity.

The Register asked van Rossum to comment for the occasion but he declined, stating that he's not all that interested in promoting himself.

Python is currently managed by the Python Steering Council, which consists of five people who serve for the duration of a feature release, the most recent of which was Python 3.9.0 last October. Currently, the group includes: Barry Warsaw, Brett Cannon, Carol Willing, Pablo Galindo Salgado, and Thomas Wouters.

These five oversee technical changes to Python and manage the community governance process – a process based on Python Enhancement Proposals (PEPs). They coordinate the contributions of more than 90 active core developers and other members of the Python community.

Python swallows Java to become second-most popular programming language... according to this index

READ MORE

For the past 20 years, as of March 6, 2021, the Python Software Foundation (PSF) has supported the language's development. Its aim is to "promote, protect, and advance the Python programming language, and to support and facilitate the growth of a diverse and international community of Python programmers."

"The PSF has a little bit been distanced from the actual maintenance of the language itself," said Ewa Jodlowska, executive director of the PSF, in an interview with The Register. "But through the creation of the Steering Council, that has created a vessel for us to be able to communicate with and work on future funding requests that might aid them in some things that they lack."

The Python Foundation has found it difficult to meet its funding goals during the COVID-19 pandemic. According to Jodlowska, much of the foundation's revenue has traditionally come from PyCon US. The event was virtual last year and will be again this year, limiting the potential income.

Jodlowska said one of the PSF's goals this year is to hire a full-time core developer, an ambition fulfilled on February 11, when Google announced plans to donate more than $350,000 to the PSF to support three projects: a malware detection system for the Python Package Index (PyPI); improvements in Python tools and services; and paying for a CPython Developer-in-Residence for 2021, to work on language maintenance.

CPython is the reference implementation of the language, written in C, the one you download from Python.org. But there are others like IronPython (C#), Jython (Java), and PyPy (RPython, a Python subset).

Asked what has made Python so successful, Jodlowska cited the importance of the Python community and the role its Code of Conduct plays as a support structure.

"Diversity for the maintainers and core developers of Python is being addressed in several ways," Jodlowska explained.

The Code of Conduct, she said, is now being enforced by the Steering Council. That's a big deal, she said, "because prior to the Steering Council existing there really was no enforcement for keeping community discussions civil and welcoming."

In theory, the Steering Council's Code of Conduct enforcement will prevent situations like the Walrus operator debate that drove van Rossum to give up his governance role. But to judge by remarks from Python Steering Council member Brett Cannon, moderating community debate simply moves the point of friction from the community to its leadership.

We are still very much a volunteer-run project, but our size demands a lot of time to keep running

Asked via email about the greatest challenge facing the Python community, Cannon said it's just trying to keep up with the size and volume of the project.

"We are still very much a volunteer-run project, but our size demands a lot of time to keep running," he said. "Tack on the usual naysayers for any decision made and it's a lot to manage both from a time and emotion standpoint."

At companies like Facebook and Microsoft that moderate the worst sorts of content, the workers who screen toxic posts and violent videos need psychological support. Managing opinionated developers in the Python community may not generate comparable levels of stress, but keeping things civil still appears to take a mental toll, particularly among volunteers.

For the Python community, bringing new people in so others can step back or delegate may help mitigate that sense of siege. Jodlowska credits efforts by Python core developers to keep the community vital. "A lot of the current core developers, for example, on their own time mentor others who are interested in becoming core developers," she said. "And there's definitely a steady stream of new incomers that way."

Blessing and a curse

Many of the current leading programming languages have strong corporate associations. Java is the child of Sun and later Oracle. Swift is mainly an Apple affair. C# is tied to Microsoft. Go and Dart arose from Google. JavaScript escaped from Netscape, which provided the browser source code that led to Mozilla, the incubator of Rust.

But Python has never enjoyed a doting or overprotective corporate parent, which according to Cannon has been both a liability and a benefit.

Apple, Microsoft, PayPal among 35 organizations compromised by evil twin dependencies attack

READ MORE

"It has hurt us as we have to go out to go searching for funding for everything and we lack paid developers to help keep things running (the best estimate we have is there's a cumulative total of about three to four devs putting in paid time on Python, and one of those is a single person with about 80 per cent time; rest is a smattering from several folks)," he said. "But being independent also means our users never have to worry about us being directed by business needs and can instead always focus on our users and their needs (with the limited resources we have). So it has pluses and minuses."

Asked about how Python goes about reconciling interest in new features with concerns about complexity, Cannon acknowledged that it's a constant challenge.

"There's always tension between expanding the language to make developers even more productive while letting it continue to 'fit your brain,'" said Cannon.

He said it's a balance of how early on would a new user likely come across a feature; the ease with which someone would recognize, if not comprehend, a feature when seen for the first time; the difficulty of searching for answer to find out how a feature works; and the extent to which a feature's function is memorable to those trying to learn it.

Python's other noteworthy obstacle is its scarcity on mobile devices. There are ways to run Python code on phones, like the Kivy framework, but Python isn't the first choice of most mobile app developers.

"I hope it will improve in the next three to five years," said Cannon. "There are several groups that are actively trying to tackle the problem from different angles, but they all require tackling big, hard problems."

Thirty years on, Python deserves recognition for what it has accomplished but it can't rest on its laurels. Rival programming languages like Julia and R in data science, and Go in cloud-native applications, have been turning heads. And the need for greater memory safety, to reduce security risks, has helped push TypeScript and Rust into the spotlight. Uneasy lies the head that wears a crown. ®

Send us news
78 Comments
Get our DevOps newsletter

More Linux love for Windows Insiders with a kernel update

Rounded corners are nice, but what you really want is Linux 5.10, right?

Windows Insiders have been given a bit of Linux love with the arrival of a freshly updated kernel and an all-important clock fix.

Having yanked the Windows Subsystem for Linux (WSL) 2 out of the usual Windows servicing cadence, Microsoft's engineers have been able to update WSL 2 without requiring a full-on OS patch.

The original 4.19 branch was updated to 5.4.72 in February. The kernel has now been brought considerably more up to date with the 5.10.16.3 version.

Continue reading

Sysadmin for FIN7 criminal cracking group gets 10 years in US prison for managing card slurping malware scam

Plus Pwn2Own faces fire and update Chrome immediately

In Brief The former systems administrator for the FIN7 card-slurping gang has been sentenced to 10 years in a US prison.

Fedir Hladyr, 35, pled guilty to one count of conspiracy to commit wire fraud and one count of conspiracy to commit computer hacking last year, and on Friday was sentenced for his role in the theft and resale of over than 20 million customer card records from over 6,500 point-of-sale terminals across the US using the malware dubbed Carbanak.

Hladyr set up a front company, Combi Security, to cover his actions as he funneled the purloined data around the criminal underworld. He managed the encrypted comms network the gang used, ran the server farms used to spread and exploit malware, and coordinated individual attacks.

Continue reading

Japanese auto chipmaker Renesas expects to resume full production next month following fab blaze

Glimmer of hope on the semiconductor front – for the car industry anyway

Japanese chipmaker Renesas has said it will restore full production capacity at its N3 Naka plant by the middle of next month following a blaze in March that destroyed equipment and contaminated the clean room.

Renesas, which accounts for a third of all automotive semiconductor sales globally, said it expects to be at half-capacity by the end of April. CEO Hidetoshi Shibata confirmed in a press conference the company plans to install new fire suppression equipment to prevent any future fires.

Operations at the Naka N3 clean room resumed on 9 April. According to a notice from Renesas, the company had to rely on over 1,600 workers each day (both internal and from third parties) to rebuild and decontaminate the clean room, illustrating both the scale of destruction and difficulty in restoration.

Continue reading

Huawei could have snooped on the Dutch prime minister's phone calls thanks to KPN network core access

Nobody caught – er, held us responsible, says Chinese firm

Huawei was able to snoop on the Dutch prime minister's phone calls and track down Chinese dissidents because it was included in the core of the Netherlands' mobile networks, an explosive news report has claimed.

Dutch national daily Volkskrant (behind a pay wall) reported over the weekend that mobile operator KPN, which used Huawei-supplied equipment in the core of its network, discovered the full extent of the Chinese company's doings in 2010 after it commissioned Capgemini to write an outsourcing risk analysis report .

Not only could the prime minister be eavesdropped on by Huawei, along with millions of other customers, said KPN as it quoted the report, but it could also identify people being snooped on by the Dutch state as well.

Continue reading

On a dusty red planet almost 290 million km away... NASA's Ingenuity Mars Helicopter flies

NASA’s JPL lab speaks to The Reg

The first human-made helicopter to take flight on another planet, Ingenuity, has hovered in Martian skies after NASA at last launched the device into the air.

Amid cheers, engineers confirmed the diminutive helicopter had spun up its rotors, taken off, landed and spun everything down, leaving the stage set for further tests. An image from the helicopter's onboard navigation camera showing its shadow on the surface of Mars was swiftly followed by another sequence from the Perseverance rover showing the helicopter hovering.

Continue reading

Oracle cuts support for South African energy biz Eskom in long-running licensing dispute

'Eskom should pay the pending dues for the Oracle software that they use'

Oracle has pulled the plug on support for software described as "quite essential" to "crucial operations" at South African energy firm Eskom as part of an ongoing licensing dispute.

Eskom spokesman Sikonathi Mantshantsha said Big Red had withdrawn support for multiple software systems after the electricity provider failed to have the courts compel Oracle to continue while the dispute was settled. Eskom had also offered to pay what it thought it owed upfront until the figure was agreed in court.

Mantshantsha confirmed that Oracle had withdrawn some of its technical support services. "Eskom has contingency plans in place to reduce the risk of disruption resulting from the dispute with Oracle," he said.

Continue reading

Plot twist! South Korean telco uses 5G to fight coronavirus via hospital-patrolling robot

Modified Keemi disinfects, takes temperatures, tells you off for not socially distancing

South Korea Telecom (SKT) has linked up with Yongin Severance Hospital to commercialise and deploy facility-roaming robots that minimise the need for face-to-face contact, thus supporting reduced COVID transmission.

"The plan is to ensure that citizens can safely use the hospital through a 24-hour constant quarantine system, and to further strengthen the infection control system in the hospital so that patients in the Corona 19 environment can receive treatment at the National Safety Hospital without anxiety of infection," said SKT in a canned statement.

The robots take temperatures via facial measurements. Mask checks are done through facial recognition, AI technology, and voice guidance warnings. Social distancing is analysed via AI technology and 3D cameras that can calculate distance. During the day, the robot offers hand-sanitising services. At night, it sterilises the environment via UV rays. Operation and other real-time data is communicated to operators over 5G.

Continue reading

UK Home Office tenders £5m for a supplier to help it greenlight IT projects. Yes, you read that correctly

Procurement raises questions over supplier creating its own sales pipeline within govt

The UK's Home Office is tendering to recruit a supplier to help manage the selection of its IT projects, leading to concerns over conflict of interest.

The notice published in the public sector Digital Marketplace is seeking a company to help deliver and operate the "discovery-as-a-service" capability for the "Innovation - Law Enforcement" (I-LE) function within the Police and Public Protection Technology Portfolio (PPPT), with a £5m contract on the table.

The snappy moniker – DaaS – alludes to the discovery phase in the UK government's IT project service manual. Discovery, it says, means learning about users and what they're trying to achieve; constraints the project faces in making changes to how the service is run because, for example, of technology or legislation; and the underlying policy intent the project is set to address and so on.

Continue reading

Brit authorities could legally do an FBI and scrub malware from compromised boxen without your knowledge

Would move for The Greater Good™ actually be good, though?

Comment UK authorities could lawfully copy the FBI and forcibly remove web shells from compromised Microsoft Exchange server deployments – but some members of the British infosec industry are remarkably quiet about whether this would be a good thing.

In the middle of last week the American authorities made waves after deleting web shells from Exchange Server deployments compromised in the Hafnium attacks. The agency had gone to the US federal courts for permission, which it received.

The entire infosec world had been bellowing at IT admins to update and mitigate the vulns, which were being exploited by skilled and malicious people who found the remote-code-execution bug. Nonetheless, some laggards still hadn't bothered – and with compromised boxen providing a useful base for criminals to launch further attacks from, evidently the FBI felt the wider risk was too great not to step in.

Continue reading

Truth and consequences for enterprise AI as EU know who goes legal: GDPR of everything from chatbots to machine learning

Regulations On A European Approach For Artificial Intelligence

One of the Brexit bonuses we’ve been enjoying since January 1st is that we have abandoned our influence within the world’s regulatory superpower.

America and China may have industrial and military dominance, but by placing a decent proportion of global economic activity under the world’s strongest regulatory regime, the EU forces the pace for everyone else. GDPR commands respect around the world.

So when the draft "Regulation On A European Approach For Artificial Intelligence" leaked earlier this week, it made quite the splash - and not just because it’s the size of a novella. It goes to town on AI just as fiercely as GDPR did on data, proposing chains of responsibility, defining "high risk AI" that gets the full force of the regs, proposing multi-million euro fines for non-compliance, and defining a whole set of harmful behaviours and limits to what AI can do with individuals and in general.

Continue reading

Debian devs decide best response to Richard Stallman controversy is … nothing

Two-week vote dismissed options to back or sack controversial FOSS figure

The Debian developer community has decided to say nothing about the new controversy surrounding Richard Stallman relection to the board at the Free Software Foundation.

The decision to say nothing came after a call for the project to support an open letter that called for Stallman’s removal from all leadership positions in the free software community and the removal of the entire Free Software Foundation for enabling Stallman.

Stallman resigned from the Foundation in 2019 after making incredibly insensitive remarks, in which he questioned whether the term “sexual assault” was applicable in the case of a woman who, aged 17, was coerced to have sex with MIT professor Marvin Minksy.

Continue reading