The quality of open source code has improved over the last two years, according to an audit sponsored by the US Department of Homeland Security.

The security and quality of more than 250 open source projects - including Apache, Linux, Firefox and PHP - was assessed using code analysis tools from Coverity as part of the federal government's Open Source Hardening Project. Coverity set up a scan site that invited individual developers to put their code through its paces with its static source code analysis tool, Coverity Prevent.

The same approach was used to analyse 250 popular open source projects, containing more than 55 million lines of code, on a regular basis. This analysis revealed a 16 per cent reduction in "static analysis defect density" across popular projects over the last two years, reflecting the discovery of 8,500 individual defects. The site divides open source projects into rungs on a ladder based on how far each project gets in fixing bugs.

‘NULL pointer dereference’* was the most common bug identified by the scans. The project discovered that larger projects are not prone to a higher density of bugs, a finding that contradicts conventional wisdom.

Static code analysis looks at source code without compiling software and executing code. Dynamic analysis, much less user experience of using open source code, was beyond the scope of the project.

Code analysis tools in general are imperfect. The scan results turned up false alerts on bugs in 14 per cent of cases.

Even taking that into account the signal to noise ratio is good, and the scan probably helped identify many bugs at an early stage. The audit is reckoned to be largest code analysis project to date. More data on the project is available through Coverity's research library (here (registration required). ®

*Buffer overflow flaws are the staple of most security bugs but experts warn that null pointer bugs could become fertile ground for hacking attacks. "Null pointer security flaws are exploitable and could quickly replace buffer overflows as the next big threat," said Geoff Sweeney, CTO of Australian-based net security firm Tier-3.

3 comments posted — Comment period finished