Boffins in Finland have scanned the open-source software libraries in the Python Package Index, better known as PyPI, for security issues and said they found that nearly half contain problematic or potentially exploitable code.
In a research paper distributed via ArXiv, Jukka Ruohonen, Kalle Hjerppe, and Kalle Rindell from the University of Turku describe how they subjected some 197,000 Python packages available through PyPI to a static analysis tool called Bandit and found more than 749,000 instances of at best poor, or at worst insecure, programming.
"Even under the constraints imposed by static analysis, the results indicate [the] prevalence of security issues; at least one issue is present for about 46 per cent of the Python packages," the researchers claimed.
Of the issues identified, most (442,373) are low severity. About 227,426 are moderate severity, present in about 25 per cent of PyPI packages. And about 80,065 are high severity, present in about 11 per cent of PyPI packages.
Of the 46 per cent of packages with issues, the median number of problems is three. But a few packages were much worse than most.
"At the very end of the tail, there are five packages with more than a thousand detected issues:
genie.libs.parser, respectively," the paper says.
Among these worst offenders, many of the detected issues were not particularly worrisome. For example, the boffins said that all of the 2,589 problems with
PyGGI have to do with the “try-except-pass” construct, which they suggested may be more an inadvisable pattern (a "code smell") than a confirmed vulnerability.
appengine-sdk, an unofficial package of Google's Python development kit for its App Engine cloud service, exhibited more serious problems. Among the 2,356 issues detected, 395 refer to generic known risky issues, 351 are injection-related, 500 have to do with potential cross-site scripting, and 7 involve potentially insecure network protocols.
"Although these observations are partially explained by the fact that
appengine-sdk embeds a large amount of third-party libraries directly to its code base, a closer look reveals numerous problematic and potentially insecure coding practices also in Google’s own code," the paper says, evidently mistaking this unofficial project for one that Google maintains.
- Thinking about upgrading to Debian Bullseye? Watch out for changes in Exim and anything using Python 2.x
- NPM is Now Providing Malware – or was until recently
- Security warning deluge from 'npm audit' is driving developers to distraction
- If you've mastered Python 101, you're probably better at programming than OpenAI's prototype Codex
Other surveys of this sort have come to similar conclusions about software package ecosystems. Last September, a group of IEEE researchers analyzed 6,673 actively used Node.js apps and found about 68 per cent depended on at least one vulnerable package.
Only last March, PyPI purged 3,653 malicious packages after someone created an account named "RemindSupplyChainRisks," apparently to show how easy it is to poison PyPI with bad packages.
The situation is similar with package registries like Maven (for Java), NuGet (for .NET), RubyGems (for Ruby), CPAN (for Perl), and CRAN (for R).
In a phone interview, Ee W. Durbin III, director of infrastructure at the Python Software Foundation, told The Register, "Things like this tend not to be very surprising. One of the most overlooked or misunderstood parts of PyPI as a service is that it's intended to be freely accessible, freely available, and freely usable. Because of that we don't make any guarantees about the things that are available there."
The library problem
Modern software tends to rely on open source libraries written by third-parties who in turn have incorporated other libraries as dependencies in their projects. Meanwhile, malware creators have recognized how much software gets trusted by default – because there's no practical way to check dozens of packages and many levels of dependencies – and have stepped up efforts to subvert the system.
Back in 2018, security vendor SonaType reported that open-source software supply chain vulnerabilities had doubled in the previous 12 months. And today, in the wake of the SolarWinds and Kaseya supply chain attacks, among others, it's clear vulnerabilities in software development systems and build tools need even greater attention.
Durbin welcomed the work of the Finnish researchers because it makes people more aware of issues that are common among open package management systems and because it benefits the overall health of the Python community.
It's not something we ignore but it's also not something we historically have had the resources to take on
"It's not something we ignore but it's also not something we historically have had the resources to take on," said Durbin.
That may be less of an issue going forward. According to Durbin, there's been significantly more interest over the past year in supply chain security and what companies can do to improve the situation.
For the Python community, that's translated into an effort to create a package vulnerability reporting API and the Python Advisory Database, a community-run repository of PyPI security advisories that's linked to the Google-spearheaded Open Vulnerability Database.
"We're really excited about the opportunities presented by the Advisory DB and the inbound API to surface [security concerns] to maintainers," said Durbin. ®