Interview Linux kernel development – which is driven by plain-text email discussion – needs better or alternative collaborative tooling "to bring in new contributors and maintain and sustain Linux in the future," says Sarah Novotny, Microsoft's representative on the Linux Foundation board.
Said tooling could be "a text-based, email-based patch system that can then also be represented in a way that developers who have grown up in the last five or ten years are more familiar with," she added.
Novotny has been at Microsoft for just over a year, working in Azure's Office of the CTO where she describes herself as an “open source wonk.” She came from Google, where she was head of open-source strategy for the web giant's Cloud Platform. “I have a broad remit to investigate and engage in open source across the company,” she told us.
Microsoft's rep on the Linux Foundation board ... Sarah Novotny
We asked Novotny for the Linux Foundation board’s perspective on a problem recently raised by Linux kernel creator Linus Torvalds, that “it’s really hard to find maintainers.” Maintainers are the gatekeepers who determine what code ends up in the widely used open-source kernel, and who ensure its quality. It is part of a wider discussion about who will oversee Linux when the current team moves on.
“The board has been involved in conversations about succession and there is work being done to build that plan,” said Novotny. Part of the answer is to bring in new contributors, and it is here that our discussion turned to the tools used for Linux development, not the C compilers and assorted toolchains, but the fact that the mechanism for submitting and discussing new code centers on plain-text email lists.
“The Linux Kernel Mailing List (LKML) is still the main tool for collaboration among kernel developers,” states the home page for Linux on the foundation site.
The email mechanism is effective, which is why it has endured, though Novotny said it is also a barrier to new contributors. “We need to set up a better or a different or an additional way to view tools and the work that’s being done in the Linux project as we’re trying to bring in new contributors and maintain and sustain Linux in the future,” she told us.
We need to set up a better or a different or an additional way to view tools and the work that’s being done in the Linux project
Picking her words carefully, she said work is being done towards “moving from a more text-based, email-based, or not even moving from, but having a text-based, email-based patch system that can then also be represented in a way that developers who have grown up in the last five or ten years are more familiar with.
"Having that ability and that perspective into Linux is one of the ways we hope we can help bring newer developers into the kernel track.”
Should it migrate toward something more like, say, issues and pull requests on the Microsoft-owned GitHub? “I’m not saying that there will be a move in any time that I can see – my crystal ball’s broken – but I do think there needs to be expansions in the way people can enter that workflow,” said Novotny.
“It is a fairly specific workflow that is a challenge for some newer developers to engage with. As an example, my partner submitted a patch to OpenBSD a few weeks ago, and he had to set up an entirely new mail client which didn’t mangle his email message to HTML-ise or do other things to it, so he could even make that one patch. That’s a barrier to entry that’s pretty high for somebody who may want to be a first-time contributor.”
We assumed that Outlook was to blame. Could Microsoft fix that instead? “The question always is fix it to whose standards, because we are focused much more on business and enterprise models of clients and customers. For them we fixed it to a more HTML-based model so it really depends on who your audience is and who your target is.”
It turned out, though, that this time Outlook was not guilty. “I think it was actually Gmail that was a barrier. And he also couldn’t do it from Apple Mail. It is just that the modern mail client has intentionally moved towards HTML,” she said.
Linux and open source at Microsoft
What is the future of Linux at Microsoft? “Linux at Microsoft continues to grow both as an investment space and a usage space,” said Novotny. “We have more than half of the Azure cores now are running Linux in some flavor.
"The teams that work on this are spending a lot of effort upstreaming any of the changes they find or make in order to make sure that we’re being good open source citizens. We have one Microsoft member on the technical advisory board for Linux. Sasha [Levin] has that role, and he and I collaborate on how do we help this move in a direction that is good for Linux,” she said.
Novotny spends much of her time as an internal consultant on open source at Microsoft: whether and how to open up code and manage projects. How does the Windows giant determine what is and is not open sourced?
“When I’m brought in as a consultant to talk about that, my first question as a total open source pragmatist is what do you think you get by being open source? What is the benefit to Microsoft in making this open source? What is the benefit to the community? We think of open source as just a tool in a toolbox, helps us move this project or this industry change further and faster,” she said.
These are business decisions, and finding the path that works for all the stakeholders is tricky, as we have seen with Google and its contortions over Kubernetes, Istio, and the controversial formation of the Open Usage Commons. Having worked at Google on open-source strategy, does Novotny have any insight into that?
I know that they’re trying to work to solve a problem that they see while still maintaining the control of the project that they feel is necessary
“I saw some of the rumblings as I was leaving, of what might be coming,” she said. “I know that they’re trying to work to solve a problem that they see while still maintaining the control of the project that they feel is necessary. That is one of the patterns they’re trying. It’s too early to reach any sort of conclusion on that.”
Novotny reflects on Kubernetes and the challenge of seeing how the community has grown so immensely, but “then to hear people say that too much was given away, that’s a hard thing to hear, because I don’t think with a very closely held open source project that we could have moved the industry as far and as fast,” she said.
With Microsoft’s open-source projects like Visual Studio Code, Dapr, and OAM (Open Application Model), Novotny said that “we want to level the playing field for people to contribute from the outside while still making sure that we can meet Microsoft’s business needs from this project. And that’s the dance that you’re talking about. That’s exactly the dance that we did within Kubernetes as well, and it depends on who you ask if we danced that well. That’s always going to be the case.”
Everything OK with Microsoft? Windows giant admits it was 'on the wrong side of history' with regard to open sourceREAD MORE
There is another aspect to open source at Microsoft, which is as a consumer. This extends both to using external open source, and also adopting more of an “inner source” model where code is shared within the corporation.
“We’ve been doing a lot of work around inner sourcing to get people more comfortable with looking to see if someone else has done this first and if you can use that or contribute to that or participate in it, before you go build your own,” she said.
The key to success, she added, is to do this “well with intention and clarity and transparency and a really good software supply security model.” This last point references the fact that open-source dependencies bring with them a risk of security issues or even malware.
The Linux Foundation is trying to address this with the newly formed OpenSSF (Open Source Security Foundation). “Now that we have won with open source we need to manage it to the standards that enterprises are used to,” said Novotny.®