Python is getting faster: Major performance tweaks on horizon

Instagram, Microsoft responsible for lifts coming in version 3.11 and beyond


The next version of the standard Python interpreter, CPython, is expected in October. It will include significant performance improvements and support for running inside the browser.

Last week, the first Python language summit since 2019 took place in Salt Lake City. At the event, the language's development team announced various changes for the forthcoming version of the language, as well as its near future. The Reg has covered some future improvements before, and as they get closer, details are becoming clear, as well as what's coming in Python 3.12.

There are multiple editions of Python out there, including interpreters for the JVM and .NET CLR, as well as compilers, but the core implementation of the language is the CPython interpreter. This has some well-known limitations, including the Global Interpreter Lock or GIL, which prevents the language from taking full advantage of multicore processors.

It has long been possible for a single process to contain several interpreters, but they interfere with one another because they all must share the single GIL. Eric Snow's effort to resolve this is the per-interpreter GIL: to give each interpreter its own GIL.

A broader solution would be to remove the GIL altogether. A previous effort to do this, Larry Hastings' GILectomy, stalled, but a new one has already gone further and may yet succeed. The new feature is simply called nogil and is being worked on by Meta developer Sam Gross.

This isn't the first time that the Facebook owner has contributed go-faster changes. Apparently, Meta owned Instagram uses Python very heavily and runs on an internal version called Cinder, but this is specialized for the company's needs and isn't for general consumption. The new effort should be more widely applicable.

These changes may be in small part due to another industry giant. Back in 2018, Python Benevolent Dictator For Life Guido van Rossum retired, but a couple of years later, he changed his mind and returned to work – at Microsoft.

With Microsoft paying the wages of Van Rossum and Mark Shannon, along with some money from Bloomberg and others, the HotPy project is making progress. So far, the beta of CPython 3.11 is on average about 25 percent faster in benchmarking.

The speedup project also has a roadmap of future changes, and the Python Software Foundation's blog explains the plans in more detail. ®

Similar topics

Broader topics


Other stories you might like

Biting the hand that feeds IT © 1998–2022