What a cluster fsck: New scheduling code plus Intel's Alder Lake CPU mix equals a slower Linux kernel 5.16
Can we just postpone these changes to next year?
The mixture of performance and efficiency CPUs in Intel's 12th-gen Core processors, code-named Alder Lake, hasn't just caused problems for some Windows gamers – it's led to complications for Linux.
This performance regression involves two related problems. What's interesting is that the origins of at least one of the issues affecting the latest Intel chips lies in a totally different architecture.
Earlier this year, Intel officially canned Lakefield, its first effort at hybrid chips – but Lakefield's Tremont efficiency cores live on. They are the basis for a range of specialist server processors, such as the network-infrastructure-oriented Atom P5900 series, codenamed "Jacobsville" and "Snow Ridge".
These are manycore chips with eight to 24 Atom cores, arranged in clusters which share resources such as cache memory. Huawei subsidiary Hisilicon's Kunpeng 920, a manycore ARM chip, shares a similar clustered design.
Which is the reason that a cluster-aware scheduling patch was submitted together by Intel and HiSilicon engineers. It makes the kernel's scheduler aware that clusters of cores have shared cache.
The goal of this on-by-default scheduling system is to balance the threads running in each cluster. Threads that share data should stay in the same cluster, while threads that don't share data can be moved to a different cluster. In synthetic benchmarks, this led to between single-digit and 25 per cent performance improvements.
- The rocky road to better Linux software installation: Containers, containers, containers
- Alpine Linux 3.15 bids a fond farewell to MIPS64 support
- AWS commits to update its own Linux every other year
- Just because you can do it doesn't mean you should: Install Linux on NTFS – on the same partition as Windows
But here's the rub. Alder Lake chips sport a mix of efficiency and performance cores, and the cluster-aware scheduler may not be fully aware of the differences between the CPU cores when selecting them to run threads. The performance cores are SMT capable, and the efficiency cores aren't.
Thus, a thread on an efficiency core in an Alder Lake system could be kept on that core for cluster-balancing reasons, when really it should migrate to a performance core, causing a dip in benchmark results.
Linux, as used in Android, already supports the Arm architecture's mixing of beefy and lightweight cores – known as big.LITTLE then DynamIQ – in things like smartphones and tablets. From what we can tell, the cluster-friendly scheduler takes a separate approach, and on x86 (at least) is good for chips like Jacobsville with uniform CPU cores, but not so good for things like Alder Lake where the cores are different types.
Compounding this headache is an issue with Intel's Turbo Boost Max 3.0 (ITMT) on some overclocked or XMP-enabled Alder Lake system motherboards, which causes all the processor cores to be identified by the Linux kernel as having the same maximum performance capabilities, which prevents the scheduler from favoring higher-performance cores for workloads.
The 12th-gen chips also have an improved version of Intel's Thread Director, a control unit that reports back to the operating system precisely how each core is performing. Windows 11 is able to tap into this intimate detail, but for now, Linux isn't – which may or may not be linked to the Alder Lake slowdown.
Release candidate 3 addresses the ITMT problem, though as Phoronix notes, "the cluster-aware scheduling may still be causing some issues with 5.16." Whatever the ultimate cause of the speed regression, it was measured on Alder Lake and traced back to its mix of P and E cores.
Again, these are release candidates, so unless you're opting to install them on an Alder Lake system, you shouldn't feel the effects of this clash between code and hardware. One solution for the kernel devs may be to disable the cluster-aware scheduling by default; another may be to disable it on Alder Lake systems, or perhaps drop it from 5.16 completely.
As usual, there are multiple other tweaks and improvements coming to the kernel, including performance improvements, better Raspberry Pi Compute Module support, and live migration of AMD's encrypted virtual machines. There's even a new kernel system call,
sys_futex_waitv(), specifically to help the performance of some Windows games played in WINE on Linux – another sign that such things are becoming more mainstream. ®