Agent P waxes lyrical about 14 years of systemd

Lennart Poettering gave packed-out keynote talk. Jack Dorsey … didn't

FOSDEM 2025 FOSDEM returned to Brussels for the first weekend in February – not without some controversial people.

"It's now the biggest FOSS event in the world, in terms of the number of streams, program items and talks, and in terms of virtual attendees," co-organizer Richard "RichiH" Hartmann told The Reg.

We spoke to him just before he delivered his FOSDEM infrastructure review, and he claimed: "Every year sets new records for how many virtual attendees stream the event, and this year, we already broke it by midday on Sunday."

It's free, and there are no badges or memberships – you just turn up – so it's hard for them to track warm bodies. (It's the geek's Glastonbury, and it's pretty difficult for attendees to get from one talk to another too.) This year, doubtless the best-insulated warm body – at least financially speaking – was Twitter, Bluesky, and Block co-founder Jack Dorsey, who had planned to give a keynote talk on Infusing Open Source Culture into Company DNA.

Many, including über-hacker Drew DeVault, whom we have quoted in The Register a few times, received this news with less than universal approbation. DeVault organized a sit-in protest and the talk was canceled. We did not see Mr Dorsey ourselves amid the teeming hordes, but FOSDEM is big enough that we didn't see lots of people we'd hoped to, even when trying to and frantically exchanging Signal messages.

Among some open-saucy circles, the figure of systemd supremo Lennart Poettering is almost as controversial – he joined Microsoft. His main-program item went ahead, titled 14 Years of systemd:

Let's have a look back at the tumultuous beginnings, how we became what we are now, and let's talk a bit about the perspective for the future, what systemd is supposed to be in the eyes of its developers – and what it shouldn't be.

He received a rapturous welcome, with a massive round of applause, and not a single rotten tomato was flung. It was a rapid, dense talk and he had to skip several of his 34-page slide deck [PDF].

Poettering started with what he called the prehistory: "Once upon a time, there was sysvinit. It's still maintained! But it's old," noting that sysvinit first appeared in 1983, and the Linux version in 1992. If that's prehistory now, then we are put in mind of other wise words: History became legend. Legend became myth. And some things that should not have been forgotten were lost.

Agent P then moved on to Canonical's Upstart, which he said was born in 2006 and was dead by 2014. He claimed it failed because:

It didn't solve the problem: admin/developer told the computer on which trigger to do what action, in order to build full tree of actions to reach some ultimate goal.

He also noted the contentious issue that Canonical requires copyright assignment, which isn't unique, but annoys some developers. (Just a day after his talk, we noticed André Machado's Why Upstart from Ubuntu Failed, which covers this part of the history in more detail.)

The talk subsequently moved on to the early history, and what became today's systemd: babykit. At the time, Poettering said, there was a trend for all daemons to be called something-kit, for example, policykit, saying that "babykit was a process babysitter." He described it as a "transactional system, the basic ideas hashed out during a flight back from Linux Plumbers Conference 2009, by Kay Sievers and yours truly." He characterized it as a "proper open source project, with no copyright assignment, LGPL version 2.1." And he nodded to some influences and prior art, not only including Upstart but also Apple's launchd, especially its Socket Activation feature, and Solaris SMF. But, he said, systemd also includes "one or two original thoughts (I think?)"

This much is relatively established and agreed, but while those who were following this stuff a decade and a half ago knew that Mac OS X and Solaris both had next-gen init systems, we're not so sure that some of the louder voices today remember this. Now we mostly see comparisons with less-ambitious replacements such as OpenRC, and it seems to us that the original motivations are being forgotten. This really is a problem. As George Santayana wrote in 1905: "Those who cannot remember the past are condemned to repeat it." In his Usenet signature in 1987, Henry Spencer amended this to: "Those who do not understand Unix are condemned to reinvent it, poorly."

This struck us because next Agent P moved on to an attempt to establish systemd's UNIX™ credentials, asking: "What the heck is UNIX even?" Perhaps the team is still stinging from the response to the systemd 256 announcement last year, which boasted "now with 42 percent less Unix philosophy." His argument runs thus:

Linux is not UNIX

UNIX: unified repository with kernel and userspace in one

Linux: everything split up, distributed ownership (aka: chaos)

Linux + systemd: still everything split up, just a little bit less.

Is this UNIX? Nope. Closer to UNIX? Yeah, probably!

Judge for yourselves if this is a fair summary. We personally disagree on the detail. As we said in 2023, we feel that Linux is a Unix, but only in ways that don't really matter in the 21st century.

He went into some detail on this aspect as it is a common criticism. For instance, questioning some dogma:

"Everything is a file"

(Is it really? Sorry, but my printer is not a file! 🙃)

Well, yes, Lennart, older ones that talk PostScript – or HP PCL, or Epson Esc/P – over a good old physical cable were nodes in /dev, and you absolutely can just cat some text to them.

"Shouldn't running services be files too"?

sysvinit: nope, why would you think so?

systemd: kinda, maybe directories at least? (cgroupfs)

This part has some persuasive force, although we see it as imposing a hefty additional layer of complexity on top of the basic Unix model. The real solution that the chaps who designed Unix came up with is not to bolt on additional process namespaces; it is to put every process in its own namespace, virtual file system and all.

The result? Well, a few slides later, Poettering went on to systemd's size and complexity:

~690K SLOC

(compare: wpa_supplicant ~460K or glibc ~1.4M)

On Fedora, full blown install is 36M bytes (compare: bash ~8M)

"It's not that bad, I think"

Well, the entire 9Front OS kernel is about 5 MB, as we described at FOSDEM last year:

The entire distribution including all sources, documents, local Git repository, and binaries is circa 530 MB for amd64.

In our opinion, Ken Thompson and Rob Pike did the more elegant job, with some guidance from Dennis Ritchie. Perhaps we're biased, though, and to be fair, we use computers running Poettering's code almost every day, and ones running 9Front about twice a year. Speaking of which, a new release of 9front, dubbed This Time DEFINITELY appeared on January 19. (We suggest you take a close look at the illustration on the cover: "Welcome, NIX. Seriously.")

Apple's Wall Street Journal ad circa 1981, which reads: Welcome IBM, Seriously – click to enlarge

Poettering was also keen to note his baby's modularity:

Suite of ~150 separate binaries

Quite modular

Most components optional, except for the core: the service manager, i.e. PID 1

(Almost) all in C

On the topic of the language, he said:

We started with the oxidation process (systemd-zram-generator)

(Not part of main repository yet, though)

Limitations in the Rust tooling remain problematic:

What so far slowed us down: we have a complex build, with many binaries and tests, with many interdependencies. Not fit for Cargo. And Meson didn't used to like Rust too much. The impedance mismatch sucked, and is not something we wanted to work around ourselves.

Criticisms aside, it's undeniable, systemd has won. He noted that by 2011 it started to become default: in Fedora since 2011, in openSUSE and Arch in 2012. Soon after, it got its own conference, systemd.conf, held in 2015 and 2016 in Berlin. Since 2017, it's been called All Systems Go. By 2019, it got a logo, and its own website. His summary was:

Where are we today: All major distros. The world runs on it.

He said there is a "vibrant community," with "six core maintainers, 60 people with commit access, and over 2,600 contributors."

Unfortunately, around this point in the talk, Poettering began to realize that he was running out of time, and as he moved to the section titled "Goals and Challenges for the Future," he started skipping more and more slides, while also speeding up, and things started to become a bit less coherent. He touched on the team's efforts toward "Boot and System Integrity," which is an area we have examined before.

He said: "All big OSes (Windows, Android + ChromeOS, Apple) have boot and system integrity implemented in one way or another," and wants to bring this to generic Linux. This is where this vulture personally starts to disagree. We don't want encrypted drives, TPM modules, sealed validated boot processes, managed home directories that aren't actual directories anymore, or pretty much any of these features – but little of that got discussed.

There were slides about the systemd efforts towards image-based OS deployment, persistent OSes, re-engineering inter-process communication, and replacing the use of dbus with something called Varlink, using JSON to pass information between programs instead of plain text – but, unfortunately, he skipped over almost all those slides.

It's clear that Agent P has a vision, or visions, for the future development of Linux and how it works. Sadly, this time around, we got more of the history and state of the art than the future. It's clear that not everyone is going to agree with this plan, but it must be conceded that so far, the world is going in that direction. We feel that efforts such as the Devuan fork of Debian lack an alternative proposal. They just want to retain the old ways, rather than proposing better alternatives. This applies equally to almost all the systemd-free distros. That's not enough to mount an effective defense. The only project we've seen that really seems to engage with much of the modernization work that systemd is doing, such as seat management, is Daniel Kolesa's Chimera Linux, which tackles it using tools brought over from FreeBSD. ®

Bootnote

To be fair, it's not all that hard to get a main-program slot in the big Janson hall. Even this vulture has spoken there – and on his first try, too. It can't have been that bad; the organizers even put him there again in 2020.