Git the news here! Code quality doesn't count for much when it comes to pull requests

Reputation matters more than code correctness, boffins claim

Git, distributed version control software used by developers to manage source code, includes a command to generate what's known as a pull request, which provides developers with a way to share changes they've made to their copy of a project with the upstream version.

For example, were you to want to contribute to Kubernetes, you might create a local copy by forking the repository. You could then craft code that improves the project somehow, and then share your changes with those maintaining the official version of the project using a pull request.

The Kubernetes maintainers will then review your pull request and, if the changes or additions are deemed worthy, accept it, thereby incorporating your changes into the version the rest of the world uses.

You might expect that the acceptance rate for this particular form of code review – there are a variety of other tools to review code, like Gerrit – would be correlated with code quality. But academic research suggests otherwise.

Bit boffins from Tampere University in Finland recently looked at whether code quality issues – code smells, antipatterns and code style violations – affect the chance a pull request will be accepted by a project maintainer.

In a preprint paper titled, "Does Code Quality Affect Pull Request Acceptance? An empirical study," submitted to the journal "Information and Software Technology," researchers Valentina Lenarduzzi, Vili Nikkola, Nyyti Saarimäki, and Davide Taibi describe their analysis of 28 Java open-source projects, which included 4.7m code quality issues in 36,000 pull requests.

Of the projects evaluated, 22 were overseen by the Apache Software Foundation. The remaining six were selected using GitHub's list of Trending Java repositories.

Among the pull requests overall, 19,293 (53.08 per cent) were accepted and 17,051 (46.92 per cent) were rejected. But the acceptance rate varied significantly among different projects. For the Apache Phoenix project, the acceptance rate was a mere 9.85 per cent; the Apache Helix project has been less selective, accepting 90.85 per cent of pull requests.

Using various machine learning techniques to evaluate the code, the researchers found code quality, as measured by the PMD software analysis tool, didn't really figure into whether or not pull requests got accepted. Being a respected community figure appears to matter more.

"Unexpectedly, code quality turned out not to affect the acceptance of a pull request at all," the researchers state in their paper. "As suggested by other works, other factors such as the reputation of the maintainer and the importance of the feature delivered might be more important than code quality in terms of pull request acceptance."

Now you know why we have bugs instead of nice things. ®