The rocky road to better Linux software installation: Containers, containers, containers
Let's be real: Everyone is trying to catch up with Apple
Analysis Linux cross-platform packaging format Flatpak has come under the spotlight this week, with the "fundamental problems inherent in [its] design" criticised in a withering post by Canadian software dev Nicholas Fraser.
Fraser wrote in a blog published on 23 November that "these are not the future of desktop Linux apps," citing a litany of technical, security and usability problems. His assertions about disk usage and sharing of runtimes between apps were hotly disputed by Will Thompson, director of OS at Endless OS Foundation a day later in a post titled: "On Flatpak disk usage and deduplication," but there is no denying it is horribly inefficient.
Most people don't care about that any more, one could argue. But they should.
The Linux world has been trying to invent a cross-platform packaging format for years, but leading contenders – the older, vendor-neutral older, AppImage format as well as Ubuntu's Snap and Fedora's Flatpak – all have serious issues.
In comparison, installing software on Windows is easy. Download an installer, run it, and you have a new app. The snags are that it means trusting unknown binaries from the internet – and that it teaches Windows users that this is fine and a perfectly normal thing to do.
Plain Win32 programs have unfettered access to your computer. This is why Microsoft invented the Windows Store: it would contain only safe, vetted, approved, "modern" apps written in "managed code". (And of course Microsoft got to keep a cut of the revenues.)
How about them Apples
It's not unfair to say that everyone is trying to catch up with Apple. Not the App Store – that's indispensable (and extremely lucrative) on iOS, but you can pretty much ignore it on a Mac, if you wish. No, the target is macOS's
.app application bundles, which macOS inherited from its 1989 ancestor NeXTstep – although they're usually delivered in a classic MacOS-style
.dmg disk image file.
Applications on macOS are a specially structured folder, containing all of the program's supporting "resources" and compiled binaries for as many CPU architectures as the app creator supports. It works pretty well, but not without snags. For instance, there's no global way to update all your apps (unless you got them from the App Store). Apps tend to be big – but that's fine, because if you can afford Macs, you can afford a big disk and fast broadband, right?
Ironically, Linux could easily have had much the same because all the functionality already exists in GNUstep, the venerable FOSS rewrite of NeXTstep's core libraries. Unfortunately, no mainstream Linux uses the GNUstep desktop, and the Étoilé project to modernise it and make it a bit more Mac-like is moribund. The superficially Mac-like Elementary OS would have been richer and more capable if its developers had started from Étoilé or GNUstep rather than its homebrewed desktop environment Pantheon*.
A lot of the developers behind Flatpak are from the GNOME and Fedora communities, or their corporate backer Red Hat, but it's a desktop-independent effort and comes installed by default on some Debian and Ubuntu derivatives. It uses Red Hat technologies such as OStree to manage binaries in a similar way to Git.
Somewhere deep within your OS is a complex directory tree full of your Flatpak applications and all their dependencies, as opposed to somewhere in full view where you can interact with it, as on macOS or GNUstep.
Rather than a macOS-style directory full of files, Ubuntu's Snap format compresses an application and all its dependencies into a single, compressed SquashFS file, which is loop-mounted as the system boots.
Flatpak and Snap have a fair bit in common. Both keep your apps inside
/var/lib (although Snap makes them visible at
/snaps, and Flatpak will let you install into your home directory if you prefer). Both require you to install a supporting framework. Both do some degree of sandboxing of apps, but aren't as secure as their publicity might lead you to believe. Both do things like run silent automatic scheduled updates in the background, in a very Windowsy fashion – bad news if you're on a metered connection. It also means that if you update your OS with a shell command, then these apps won't be included.
The AppImage format has similar pros and cons to macOS' app bundles – such as lacking a global update mechanism – because it grew out of tech of a similar age. AppImage's developer took the ROX Desktop's AppDirs and put each one inside a SquashFS (which probably inspired the Snap developers). ROX and AppDir are FOSS recreations of Acorn's RISC OS, which appeared slightly before NeXTstep in 1987. It needs no supporting frameworks and you can keep your AppImages anywhere you like.
All three share a weakness in that they include almost all of an application's supporting libraries and other dependencies in its package, so packages tend to be very large – in the order of hundreds of megabytes – and so do updates. Installing large Linux apps from the command line generally takes in the region of seconds to tens of seconds, but installing a Snap or Flatpak, even on a fast connection, can take many minutes – and of course ignores any local mirrors you may have configured for your distro's built-in package manager.
- You GNOME it: Windows and Apple devs get a compelling reason to turn to Linux
- helloSystem: Pre-alpha FreeBSD project chases simplicity and elegance by taking cues from macOS
- Snapping at Canonical's Snap: Linux Mint team says no to Ubuntu store 'backdoor'
- Reviving a classic: ThinkPad modder rattles tin to fund new motherboard for 2008's T60 and T61 series of laptops
Endless's Thompson makes a good point, though. Flatpak's format allows something single-file formats can't do: if files in different Flatpaks are identical, OStree can reduce duplication by hard-linking them together… although in principle, a smart enough filesystem could do the same at the block level.
There are alternative cross-distro systems. Several avoid installing apps into the OS at all, and just fetch them off the internet into your home directory when needed. Examples include 0install, from the developer of the ROX Desktop. That inspired AppFS – which does something similar to CERN's unrelated CernVM-FS. And then there are functional package managers, which are a whole other type of software-management tool which we'll come back to in another article. Suffice it to say there are multiple vendor-neutral ways of distributing Linux software that predate the big three. All of them are lighter-weight, more efficient, the packages are generally much smaller, and all are seriously obscure and you'll never encounter them in a mainstream distro.
Naturally, because of the rampant Not Invented Here Syndrome of the Linux industry, all of these systems totally ignore one another. Which does at least have the benefit that you can install most of them side-by-side on the same OS and try them out, with no real penalties except using a lot of disk space. But at least that's cheap these days.
There are so many alternatives vying for space that it's hard to pick winners. This is partly due to rivals building their own tools rather than cooperating, and partly because there's almost no money to be made from desktop Linux, only servers – so there's little investment, and engineers' occasionally sketchy prototypes end up getting shipped.
What does seem likely is that the lean, efficient but daringly unconventional tech won't go anywhere, while the inefficient and space-hungry variants will be pushed heavily and widely used.
This being so, AppImage probably won't get much bigger, because it doesn't have a big company behind it. Ubuntu's Snap system has some advantages over GNOME's Flatpak, such as being useful on servers and so on… but Ubuntu runs the only Snap Store, and the only open-source back-end is obsolete and has been deleted.
With Red Hat's considerable clout behind it, Flatpak's chances look good. But whoever wins, in time we'll probably have to get used to distros occupying terabytes of disk and need hundreds of gigs of regular updates… and the memories of the small, efficient systems that went before will be lost to history. Isn't progress great? ®
* An earlier version of this piece mistakenly stated the OS was based on Gnome 3.