This article is more than 1 year old
Securing open-source code isn't going to be cheap
Devs and maintainers are getting paid ... though not to concentrate on security
Column Open-source software has always been more secure than proprietary software, but that doesn't mean it's "secure." To lock it down, we need to invest serious cash in developers and maintainers.
You may have noticed that a lot of people are getting seriously cranky about open-source software security lately. They have a reason. Our screw-ups have been making the news a lot lately.
To name but a few, there was the ongoing Log4j vulnerability fixups; the npm bad code injection fiasco; and you haven't heard the last of the Linux PolKit security hole since many embedded systems will never be patched.
So, what can we do about it? First, as I like to remind people, if you think open-source security is bad, that's only because you never hear about the security by obscurity blunders of proprietary software until they explode or they're patched. Just look at Microsoft's endless Patch Tuesday problems where the fixes can be as bad as the original problems.
But, just because closed-source developers have their problems doesn't take away from our foul-ups. On the open-source side of things, we need to do better soon.
Sure, as open-source co-founder Eric S Raymond pointed out in Linus' law: "Given enough eyeballs, all bugs are shallow." But it requires eyeballs looking for bugs in the first place to make it work. As Jim Zemlin, the Linux Foundation (LF)'s executive director, said in the aftermath of the Heartbleed and Shellshock security fiascos: "In these cases, the eyeballs weren't really looking."
Guess what? Things haven't changed that much. And, thanks to our ever more complicated and automated software supply chain, when we miss a bug today the pain they can cause is ever so much greater. Thanks to our modern continuous integration/continuous deployment (CI/CD) production pipelines, a problem in one tiny library today can be deployed in thousands of programs tomorrow.
I'm not exaggerating. Programmer Marak Squires deliberately added an endless loop into his colors.js package and it was bundled into almost 19,000 other npm packages and was being downloaded 23 million times a week.
Now we are making progress. The LF, OpenSSF, SPDX, and OpenChain have all been working on securing the software supply chain with Software Bills of Materials (SBOMs), and a standard for them: ISO Standard 5962. If we're to keep junk out of our CI/CD pipelines we've no choice but to adopt SBOMs.
The LF and friends are also working on spotting "new, as-yet-undiscovered vulnerabilities in open-source code" in the Alpha-Omega Project. Once found these will be fixed.
All of this is great. But, even though Microsoft and Google have contributed an initial investment of $5 million to the Alpha-Omega Project, it's not enough.
- Open-source Kubernetes tool Argo CD has a high-severity path traversal flaw: Patch now
- Open source isn't the security problem – misusing it is
- Linux Foundation spends 20% more in 2021, highlights new LFX platform
If there's one thing I've learned about programmers in the years I've been in tech it's this: We are awful at security. We're not taught in schools. Most of us put security on the back burner along with documentation. Adding insult to injury, it's seldom a priority in our jobs, whether it's a project of love we labor over in our flat or our company's Big Freaking Project of 2022.
Let's face it, does the project manager want it done on time or does she want it to be secure? Yeah, you know what Mahogany Row is going to say.
We must invest in securing our programs and stop just giving security lip service. That means making security a real priority in our projects. It means learning about how to do security. And it means paying open-source developers and maintainers to secure their projects.
Now there is an urban legend that open-source developers don't get paid. Please. We came out of the basement a long, long time ago.
As Matthew Wilson, noted open-source leader and AWS VP tweeted recently, Apache Software Foundation (ASF) "volunteers are often paid very well for their work by companies that define job duties to include building and maintaining software housed at the ASF. 'Voluntary' != 'Unpaid.'"
I don't know everyone who's anyone in Linux and open-source circles, but I know a heck of a lot of them. Almost all of them are being paid very well indeed.
It's true that some important, indeed vital, projects do fall through the cracks. But, the LF has been addressing pay for these developers. For example, David A Wheeler, the LF's director of Open Source Supply Chain Security, said the LF and its related foundations and projects are directly funding people to do security work and the LF is spending more money than ever before on security. Here's how it works.
When a problem is found, a developer reaches out to the appropriate LF organization. A contract that briefly describes what problem needs to be fixed and how it will be done, the funds required for it, and who will do the work is set up. The proposal is then examined by the appropriate LF technical review point of contact.
But – and this is an important "but" – we still don't pay enough for security. Whether you're a distinguished engineer closing in on a seven-figure compensation package or a still wet-behind-the-ears noob programmer, unless the word "security" is in your title, you probably aren't writing secure code. Fast code, yes. Good code, you bet! Secure code? Not so much.
So, when Jack Aboutboul, VP of Products at the software chain security company, CodeNotary recently asked "Did you hear about the Fortune 500 corporation that emailed an open-source maintainer to "Demand Answers" about his software package?" He's making a valid point.
Companies are demanding security from our open-source codes in new ways. Part of what we must do going forward is to continue to educate people about what open source really is.
There are far too many people and companies who assume that open-source software comes with some kind of support and service level agreement contract. Spoiler alert: It doesn't. Never has. Never will. If that's what they want, we need to tell them you can get that, but you'll need to pay for it from an open-source company. For example, if you want guarantees with your Linux, you must talk to Canonical, Red Hat, or SUSE.
Another part of it is that since people are expecting more security from our code, then we do need to think about both teaching developers and maintainers about security and paying them for their security work.
I've known many open-source programmers, and they're often truly great coders with brilliant minds, but what they're not, are security experts. They need help, education, and cold hard cash encouragement to help them fundamentally improve open-source security.
This won't happen overnight. It won't happen over the next year. But we are on our way to making these kinds of fundamental security improvements. We just need to buckle down and pay the money needed to make this happen sooner rather than later. ®