openSUSE finds an elegant solution to x86-64 version support
Piggybacking on the hwcaps tunable in glibc, it's shipping platform-optimized libraries
Updated SUSE, and the openSUSE project it sponsors, has a way around the issue of optimizing its distro for specific versions of the x86-64 architecture.
This new move was announced last week and will, hopefully, resolve the issues over x86-64 support that have been causing dissent in the distros' communities. Back in July we reported that SUSE's new ALP distro might need x86-64-v3. Then, later, the rolling-release Tumbleweed distro considered requiring x86-64-v2. Apparently, though, enough users still ran older kit that didn't support v2 and complained that the project leaders backed down and decided the new requirement would be dropped.
The resolution uses a new feature called Hardware Capability Tunables or hwcaps in the GNU C Library
glibc. The feature was introduced in glibc v2.33. Here's how the release notes described it:
The dynamic linker loads optimized implementations of shared objects from subdirectories under the
glibc-hwcapsdirectory on the library search path if the system's capabilities meet the requirements for that subdirectory.
Initially supported subdirectories include…
x86_64-linux-gnu… The subdirectory names correspond to the vendor-independent x86-64 microarchitecture levels defined in the x86-64 psABI supplement [PDF].
The first implementation of this in openSUSE was in a SUSE Hackweek project by developer Antonio Larrosa, who also explained how to use it.*
The gist is that while the main Tumbleweed distro will still be compiled for the most basic level of x86-64 support,
x86-64-v1, this new feature in glibc enables libraries to use newer CPU features if they'd benefit from them. If a particular library would be substantially faster thanks to specific features from newer versions of the processor architecture, then its developer can provide different versions of the library: fast ones that use the new feature, and slower ones that don't and have to work around it in software on older CPUs that lack the instructions.
The best part is that this is automatic. Glibc will automatically load the appropriate libraries on CPUs that match that revision of the specification: the program calling the library doesn't need to know or care. The change was accepted a few weeks ago, so any recently updated Tumbleweed installations already have it.
Some of this springs from Intel's AVX-512 feature, especially in the recent "Sapphire Rapids" Xeon Scalable processors, which The Reg discussed in January. Those new instructions were also going to be in the "Alder Lake" desktop processors, but Intel decided to disable them.
- The quest to make Linux bulletproof
- Debian-based siduction gains snapshots, new recovery tools and more
- Fancy climbing into ALP over New Year's? Fresh preview versions of SUSE's distro and NetBSD 10 are here
- openSUSE Tumbleweed team changes its mind about x86-64-v2
AVX-512 is part of x86-64-v4, and some vendors seem keen on it, but for now, SUSE is just providing some accelerated libraries that use x86-64-v3 features. At first, only a few programs are getting support, such as Python 3.11, XML and PNG parsing, and a decoder for the new JPEG-XL image format which Google chose to drop in Chrome 110 and later.
Now that the support is available, though, adoption will surely grow. In principle, developers could provide four different sets of performance-optimized libraries, one for each version of x86-64, with increasing levels of exploitation of advanced CPU features. ®
Updated on 13 March to add:
* SUSE requested that we clarify that the production version uses adifferent implementation, one which doesn't need any modifications to the packages involved. The company is still working on documenting the version that went into production.