About half of Python libraries in PyPI may have security issues, boffins say

Coding lingo's community says it has a plan to mitigate supply chain vulnerabilities, though


Boffins in Finland have scanned the open-source software libraries in the Python Package Index, better known as PyPI, for security issues and said they found that nearly half contain problematic or potentially exploitable code.

In a research paper distributed via ArXiv, Jukka Ruohonen, Kalle Hjerppe, and Kalle Rindell from the University of Turku describe how they subjected some 197,000 Python packages available through PyPI to a static analysis tool called Bandit and found more than 749,000 instances of at best poor, or at worst insecure, programming.

"Even under the constraints imposed by static analysis, the results indicate [the] prevalence of security issues; at least one issue is present for about 46 per cent of the Python packages," the researchers claimed.

Of the issues identified, most (442,373) are low severity. About 227,426 are moderate severity, present in about 25 per cent of PyPI packages. And about 80,065 are high severity, present in about 11 per cent of PyPI packages.

Of the 46 per cent of packages with issues, the median number of problems is three. But a few packages were much worse than most.

"At the very end of the tail, there are five packages with more than a thousand detected issues: PyGGI, appengine-sdk, genie.libs.ops, pbcore, and genie.libs.parser, respectively," the paper says.

Among these worst offenders, many of the detected issues were not particularly worrisome. For example, the boffins said that all of the 2,589 problems with PyGGI have to do with the “try-except-pass” construct, which they suggested may be more an inadvisable pattern (a "code smell") than a confirmed vulnerability.

But appengine-sdk, an unofficial package of Google's Python development kit for its App Engine cloud service, exhibited more serious problems. Among the 2,356 issues detected, 395 refer to generic known risky issues, 351 are injection-related, 500 have to do with potential cross-site scripting, and 7 involve potentially insecure network protocols.

"Although these observations are partially explained by the fact that appengine-sdk embeds a large amount of third-party libraries directly to its code base, a closer look reveals numerous problematic and potentially insecure coding practices also in Google’s own code," the paper says, evidently mistaking this unofficial project for one that Google maintains.

Other surveys of this sort have come to similar conclusions about software package ecosystems. Last September, a group of IEEE researchers analyzed 6,673 actively used Node.js apps and found about 68 per cent depended on at least one vulnerable package.

Only last March, PyPI purged 3,653 malicious packages after someone created an account named "RemindSupplyChainRisks," apparently to show how easy it is to poison PyPI with bad packages.

The situation is similar with package registries like Maven (for Java), NuGet (for .NET), RubyGems (for Ruby), CPAN (for Perl), and CRAN (for R).

In a phone interview, Ee W. Durbin III, director of infrastructure at the Python Software Foundation, told The Register, "Things like this tend not to be very surprising. One of the most overlooked or misunderstood parts of PyPI as a service is that it's intended to be freely accessible, freely available, and freely usable. Because of that we don't make any guarantees about the things that are available there."

The library problem

Modern software tends to rely on open source libraries written by third-parties who in turn have incorporated other libraries as dependencies in their projects. Meanwhile, malware creators have recognized how much software gets trusted by default – because there's no practical way to check dozens of packages and many levels of dependencies – and have stepped up efforts to subvert the system.

Back in 2018, security vendor SonaType reported that open-source software supply chain vulnerabilities had doubled in the previous 12 months. And today, in the wake of the SolarWinds and Kaseya supply chain attacks, among others, it's clear vulnerabilities in software development systems and build tools need even greater attention.

Durbin welcomed the work of the Finnish researchers because it makes people more aware of issues that are common among open package management systems and because it benefits the overall health of the Python community.

It's not something we ignore but it's also not something we historically have had the resources to take on

"It's not something we ignore but it's also not something we historically have had the resources to take on," said Durbin.

That may be less of an issue going forward. According to Durbin, there's been significantly more interest over the past year in supply chain security and what companies can do to improve the situation.

For the Python community, that's translated into an effort to create a package vulnerability reporting API and the Python Advisory Database, a community-run repository of PyPI security advisories that's linked to the Google-spearheaded Open Vulnerability Database.

"We're really excited about the opportunities presented by the Advisory DB and the inbound API to surface [security concerns] to maintainers," said Durbin. ®

Similar topics


Other stories you might like

  • Chip shortage forces temporary Raspberry Pi 4 price rise for the first time

    Ten-buck increase for 2GB model 'not here to stay' says Upton

    The price of a 2GB Raspberry Pi 4 single-board computer is going up $10, and its supply is expected to be capped at seven million devices this year due to the ongoing global chip shortage.

    Demand for components is outstripping manufacturing capacity at the moment; pre-pandemic, assembly lines were being red-lined as cloud giants and others snapped up parts fresh out of the fabs, and the COVID-19 coronavirus outbreak really threw a spanner in the works, so to speak, exacerbating the situation.

    Everything from cars to smartphones have been affected by semiconductor supply constraints, including Raspberry Pis, it appears. Stock is especially tight for the Raspberry Pi Zero and the 2GB Raspberry Pi 4 models, we're told. As the semiconductor crunch shows no signs of letting up, the Raspberry Pi project is going to bump up the price for one particular model.

    Continue reading
  • Uncle Sam to clip wings of Pegasus-like spyware – sorry, 'intrusion software' – with proposed export controls

    Surveillance tech faces trade limits as America syncs policy with treaty obligations

    More than six years after proposing export restrictions on "intrusion software," the US Commerce Department's Bureau of Industry and Security (BIS) has formulated a rule that it believes balances the latitude required to investigate cyber threats with the need to limit dangerous code.

    The BIS on Wednesday announced an interim final rule that defines when an export license will be required to distribute what is basically commercial spyware, in order to align US policy with the 1996 Wassenaar Arrangement, an international arms control regime.

    The rule [PDF] – which spans 65 pages – aims to prevent the distribution of surveillance tools, like NSO Group's Pegasus, to countries subject to arms controls, like China and Russia, while allowing legitimate security research and transactions to continue. Made available for public comment over the next 45 days, the rule is scheduled to be finalized in 90 days.

    Continue reading
  • Global IT spending to hit $4.5 trillion in 2022, says Gartner

    The future's bright, and expensive

    Corporate technology soothsayer Gartner is forecasting worldwide IT spending will hit $4.5tr in 2022, up 5.5 per cent from 2021.

    The strongest growth is set to come from enterprise software, which the analyst firm expects to increase by 11.5 per cent in 2022 to reach a global spending level of £670bn. Growth has fallen slightly, though. In 2021 it was 13.6 per cent for this market segment. The increase was driven by infrastructure software spending, which outpaced application software spending.

    The largest chunk of IT spending is set to remain communication services, which will reach £1.48tr next year, after modest growth of 2.1 per cent. The next largest category is IT services, which is set to grow by 8.9 per cent to reach $1.29tr over the next year, according to the analysts.

    Continue reading
  • Memory maker Micron moots $150bn mega manufacturing moneybag

    AI and 5G to fuel demand for new plants and R&D

    Chip giant Micron has announced a $150bn global investment plan designed to support manufacturing and research over the next decade.

    The memory maker said it would include expansion of its fabrication facilities to help meet demand.

    As well as chip shortages due to COVID-19 disruption, the $21bn-revenue company said it wanted to take advantage of the fact memory and storage accounts for around 30 per cent of the global semiconductor industry today.

    Continue reading
  • China to allow overseas investment in VPNs but Beijing keeps control of the generally discouraged tech

    Foreign ownership capped at 50%

    After years of restricting the use and ownership of VPNs, Beijing has agreed to let foreign entities hold up to a 50 per cent stake in domestic VPN companies.

    China has simultaneously a huge market and strict rules for VPNs as the country's Great Firewall attempts to keep its residents out of what it deems undesirable content and influence, such as Facebook or international news outlets.

    And while VPN technology is not illegal per se (it's just not practical for multinationals and other entities), users need a licence to operate one.

    Continue reading
  • Microsoft unveils Android apps for Windows 11 (for US users only)

    Windows Insiders get their hands on the Windows Subsystem for Android

    Microsoft has further teased the arrival of the Windows Subsystem for Android by detailing how the platform will work via a newly published document for Windows Insiders.

    The document, spotted by inveterate Microsoft prodder "WalkingCat" makes for interesting reading for developers keen to make their applications work in the Windows Subsystem for Android (WSA).

    WSA itself comprises the Android OS based on the Android Open Source Project 1.1 and, like the Windows Subsystem for Linux, runs in a virtual machine.

    Continue reading
  • Software Freedom Conservancy sues TV maker Vizio for GPL infringement

    Companies using GPL software should meet their obligations, lawsuit says

    The Software Freedom Conservancy (SFC), a non-profit which supports and defends free software, has taken legal action against Californian TV manufacturer Vizio Inc, claiming "repeated failures to fulfill even the basic requirements of the General Public License (GPL)."

    Member projects of the SFC include the Debian Copyright Aggregation Project, BusyBox, Git, GPL Compliance Project for Linux Developers, Homebrew, Mercurial, OpenWrt, phpMyAdmin, QEMU, Samba, Selenium, Wine, and many more.

    The GPL Compliance Project is described as "comprised of copyright holders in the kernel, Linux, who have contributed to Linux under its license, the GPLv2. These copyright holders have formally asked Conservancy to engage in compliance efforts for their copyrights in the Linux kernel."

    Continue reading
  • DRAM, it stacks up: SK hynix rolls out 819GB/s HBM3 tech

    Kit using the chips to appear next year at the earliest

    Korean DRAM fabber SK hynix has developed an HBM3 DRAM chip operating at 819GB/sec.

    HBM3 (High Bandwidth Memory 3) is a third generation of the HBM architecture which stacks DRAM chips one above another, connects them by vertical current-carrying holes called Through Silicon Vias (TSVs) to a base interposer board, via connecting micro-bumps, upon which is fastened a processor that accesses the data in the DRAM chip faster than it would through the traditional CPU socket interface.

    Seon-yong Cha, SK hynix's senior vice president for DRAM development, said: "Since its launch of the world's first HBM DRAM, SK hynix has succeeded in developing the industry's first HBM3 after leading the HBM2E market. We will continue our efforts to solidify our leadership in the premium memory market."

    Continue reading

Biting the hand that feeds IT © 1998–2021