License to thrill: Ahead of v13.0, the FreeBSD team talks about Linux and the completed toolchain project that changes everything
'For many ... vendors, the BSD license is very important compared to the GPL'
It's not as well known as Linux, but FreeBSD has plenty of hardcore fans. In a wide-ranging chat covering licensing, architectures including RISC-V, and a development model that's free of a "dictator", The Reg spoke to members of the project about new release features and more.
FreeBSD 13.0 has just reached release candidate 1 and is scheduled to come out at the end of March – with key new features including a complete LLVM toolchain, faster networking, and improved ZFS file system.
Major new releases come every two years or so: 12.0 was pushed out in December 2018, and 12.2 in October 2020. We spoke to kernel developer John Baldwin and Ed Maste, who is a FreeBSD committer and director of Project Development for the FreeBSD Foundation.
BSD stands for Berkeley Software Distribution, a discontinued version of Unix which first appeared in 1977, with FreeBSD being an open-source version, originally released in late 1993. Unlike Linux, it is not primarily licensed under GNU Public License (GPL), which is a copyleft license (meaning it insists that derivative works have the same license), but under the more relaxed BSD license – though it does include some GPL code.
"Part of FreeBSD's strength, part of what makes us unique, is our license," Baldwin told us. "A lot of people consume FreeBSD as a toolkit to build things, like appliances. For many of those vendors, the BSD license is very important compared to the GPL license, particularly GPLv3. We never imported any v3 toolchains into our native toolchain."
The updated toolchain in FreeBSD 13 was the first thing Baldwin and Maste pointed to when it comes to fresh items in the release. The big deal is that a long-term project to migrate to the LLVM compiler is now complete – and LLVM is licensed under Apache 2.0, which is less restrictive than GPL.
"Our efforts on toolchain work started about 10 years ago," said Maste. "We've been working on migrating to Clang and the LLVM family of project components. We imported Clang into the base system and migrated to it some time ago, but for FreeBSD 13 we've finished that long project. We have Clang as the compiler, LLD as the linker, and ELF tool chain as the miscellaneous binary utilities.
"Some of the tier 2 architectures, like MIPS, were still using a gcc [C/C++ compiler] 4.2.1 which is years and years out of date."
The implications of this migration extend to all the architectures FreeBSD supports, even though most architectures already used LLVM. "This is the culmination of a process to get us to a modern toolchain that also is fully BSD licensed, or a compatible license," said Baldwin.
We're looking forward to a future dominated by 64-bit x86 and Arm, and probably RISC-V...
"For architectures that weren't using Clang yet their basic code was quite old and didn't support modern C++ standards so it meant when we were writing code for base utilities we had to limit ourselves to lowest common denominator. We weren't free to use, for example, C++ 11 features. Now we have the freedom to do that."
"The old components introduced a tax or a penalty on all of the architectures; it limited what we could do on x86 or ARM because we had to carry the support for the other architectures," said Maste.
LLVM is also popular in the academic and research world. "It's a lot easier to do research with LLVM because it's designed for that," said Baldwin. Another use case is for RISC-V, which looks likely to be a key future architecture for FreeBSD. "The only way we could get a compiler for RISC-V was to use Clang. That's fully working in 13."
Regarding the RISC-V port, Maste added: "It works. It's still in the early stages of turning it into a fully usable environment, but the base system, the kernel, the userland, everything is there and works."
What else is new in 13?
"There's been a lot of work on networking performance, a few things like the FreeBSD Foundation-funded work to improve the performance of our virtual bridge interface, for VMs or some other configurations. We have a five times performance improvement.
"The PF (packet filter) firewall received a bunch of Improvements, funded by an ISP that uses FreeBSD. The virtual networking layer also has new functionality. It was experimental for a long time but has been made available by default because it's reached a stability and feature completion. Another is the in-kernel TLS work," said Maste.
In-kernel TLS? "One of the projects Netflix proposed was kernel TLS so you are able to allow the kernel to manage the encryption part of TLS, using software or in some cases we have drivers for NIC cards that can do the encryption as they are transmitting the packets out on the wire. Moving that into the kernel can give you much better throughput and performance. This is how Netflix is able to push a hundred or more Gbps of TLS traffic off a single server," said Baldwin.
Another key feature is the OpenZFS file system. "FreeBSD has included ZFS in the base system for several years. But ZFS, its development has changed over time. Developer mindshare moved more towards the ZFS on Linux tree, that's now called OpenZFS. In 13 the ZFS implementation in FreeBSD has switched to be the new OpenZFS tree. This means we have some features that weren't present in the last implementation. We are now connected to the right upstream pipeline," said Maste.
ZFS is the default file system, as we saw when taking the latest RC for a spin.
On the software side, "We've put a lot of effort into the Linux emulation layer that we have, the Linuxulator," said Maste. "Quite a number of contemporary Linux binaries will run on the FreeBSD kernel now. People have been using this for running Linux Chrome, for Widevine DRM to watch Netflix content or running some Steam games."
How is performance? "For the vast majority of Linux applications the performance overhead is effectively nil. They are either going to run just as fast or not run at all," he told us.
Is the FreeBSD team interested in using safer languages like Rust to improve security? Baldwin did not see this as the primary route to more memory safety. "One of the things I work on is a project at Cambridge called CHERI (Capability Hardware Enhanced RISC Instructions) that allow us to push special memory safety into the processor itself, so that you're able to have a much safer version of C itself.
"And that will compose well with languages like Rust. It means we can take the safeness of the language and enforce it further down the stack than we currently can. So my personal bias is I think CHERI is going to be a better solution. FreeBSD is largely written in C and we were able to run the whole base system on CHERI. So we have all of that C code running as a safer C. I think that's the more likely future way to get memory-safe systems."
One big difference [from Linux] is that we don't have one big dictator
How does the development process for FreeBSD differ from that for Linux? "One big difference is that we don't have one big dictator. We have a group of folks who have built reputation based on the work they've done," said Baldwin.
"We operate largely on mailing lists in the same way that the Linux kernel mailing list does," said Maste. "A big difference is that the FreeBSD base system is a mono repo, the kernel and the C library and the toolchain, all the base system utilities like ls and the shell are all in one single repository, built and released together.
"There is a lot of third-party contributed code; the LLVM toolchain is a fairly large portion of what makes up the FreeBSD base system. But it's quite easy for us to make a kernel change and the C library support for that together."
This also means that FreeBSD is not dependent on distros in the same way as Linux. "We don't require a distro in the FreeBSD world and the FreeBSD view of things. It is entirely possible to take FreeBSD and run it on your server or your laptop just as FreeBSD. That's the case for most of the FreeBSD developers. My laptop is running FreeBSD and I'm not using a third-party distro," said Maste.
Despite this, FreeBSD distros do exist, such as the helloSystem project, which provides a Mac-like GUI. "We're happy to have people who have an opinionated take on things to build on FreeBSD," said Maste. "helloSystem definitely got a lot of interest, in part because it is particularly opinionated, the developer of helloSystem is not trying to be everything to everyone. That's an excellent place for a distro in FreeBSD to exist." He added: "There are others like GhostBSD and NomadBSD that have been taking that approach for quite a while."
helloSystem: Pre-alpha FreeBSD project chases simplicity and elegance by taking cues from macOSREAD MORE
Does FreeBSD on the desktop matter, or is it hopeless because of the difficulty with drivers for modern hardware, an area where even Linux struggles at times?
The desktop is important, the team said, in part because it brings in new developers, and the driver situation is not as bad as it first appears. "We have developed a way of taking Linux drivers and recompile them natively under FreeBSD. We're able to track very close and have compatibility on a par with modern Linux kernels," said Baldwin.
Maste added: "The driver authors specifically support that use case. And the graphics stack on Linux is for the most part dual-licensed so the Intel graphics drivers for example are available under your choice of GPL or BSD license. We're able to use the exact same code."
"The Wi-Fi situation is not in as good shape. We're looking at both improvements to the existing bespoke drivers, and also trying to use the same model of using dual-licensed implementations in Linux and adapting them. Over the course of this year I'm hoping we'll be in the same situation with wifi as with graphics," said Maste.
What is the business model underlying FreeBSD?
"The development work comes from a few different sources. There's a lot that comes from companies like Netflix, which has a team of FreeBSD experts in-house that develop features with work that goes directly into upstream FreeBSD. There's a lot of work that comes out of academic sources. There's some work that comes from hobbyists. There's work, increasingly of late, being taken on by FreeBSD Foundation. The foundation is a US-registered charity funded by donations and has a budget of a little over $1m per year. We receive donations from individuals, and from companies that use FreeBSD."
The foundation is significant, said Baldwin, because it is bringing more balance to FreeBSD development. "Historically development has been driven by companies using FreeBSD in some fashion, whether it was Yahoo! or Netflix or storage vendors building appliances. They are going to work on the parts that work well for them. There are some gaps, that are maybe not of interest to appliance vendors but are still important for maintaining a viable ecosystem for developers. So things like improved support for laptops is one of the foundation's focuses. That's not something Netflix cares about but it is important for FreeBSD to continue to be a useful experience for our developers."
Does Apple contribute anything?
"They are not super active in contributing to upstream in the same way that Netflix is. But I don't fault them for that. We are a BSD licensed project, we're not GPL. Kirk [McKusick, a key contributor since the beginning] described the BSD license as copy centre, which is do whatever you want, we don't care," said Baldwin.
Maste added: "Apple contributes a lot to the LLVM project that benefits us immensely. Apple did a lot of work on LLVM on ARM platforms, that was important to us. I did a lot of work with LLD and Apple engineers were very forthcoming with code reviews and advice on trying to integrate LLVM support." The same is true of Sony. "Sony did a lot of work on maturing LLD to be a viable linker. It wasn't a direct contribution but it really helped us," said Baldwin.
"In the last few years we've had a bit of self-reckoning. We were trying to pivot a little bit, to be a little more future-focused. We've removed a lot of old drivers, like ISA cards. 32-bit x86, it's still there but no longer fully supported in the same way as 64-bit. We're gracefully winding it down. We're looking forward to a future dominated by 64-bit x86 and Arm, and probably RISC-V.
"It's phenomenal to me that code I've written in the last 20 years gets used in all sorts of places that I would not have thought of. My son's PS4 runs my code all the time. That's just bizarre. That's great." ®