Running on Intel? If you want security, disable hyper-threading, says Linux kernel maintainer
Speculative execution bugs will be with us for a very long time
Linux kernel dev Greg Kroah-Hartman reckons Intel Simultaneous Multithreading (SMT) - also known as hyper-threading - should be disabled for security due to MDS (Microarchitectural Data Sampling) bugs.
Kroah-Hartman, who was speaking at the Open Source summit in Lyons, has opened up on the subject before. "I gave a talk last year about Spectre and how Linux reacted to it," he told The Reg. "And then this year it's about things found since the last talk. It's more and more of the same types of problems.
"These problems are going to be with us for a long time; they're not going away."
There is another issue, though. "People didn't realise how we do security updates, the whole CVE mess, and the best practices we need to have. Linux isn't less secure or more secure than anything else. The problem is: these are bugs in the chips. We fix them in time, we just have to make sure that everybody updates."
Flushing buffers takes time. Every single one of these mitigations, to solve hardware bugs, slows down your machine ...
Kroah-Hartman explained to attendees how these vulnerabilities "exploit bugs in the hardware when the chip is trying to look into the future".
He added: "MDS is where one program can read another program's data. That's a bad thing when you are running in a shared environment such as cloud computing, even between browser tabs.
"You can cross virtual machine boundaries with a lot of this. MDS exploits the fact that CPUs are hyper-threaded, with multiple cores on the same die that share caches. When you share caches, you can detect what the other CPU core was doing."
OpenBSD was right, he said. "A year ago they said disable hyper-threading, there's going to be lots of problems here. They chose security over performance at an earlier stage than anyone else. Disable hyper-threading. That's the only way you can solve some of these issues. We are slowing down your workloads. Sorry."
Kroah-Hartman, a kernel maintainer, described some post-Spectre MDS examples, like RIDL, Fallout and Zombieland. "You can steal data across applications. You can steal data across virtual machines. And across 'secure enclaves', which is really funny. Inside Intel chips there is something called SGX [Software Guard Extensions] where you can run code that nobody else can see, it's really porous. In the kernel we fix this by flushing buffers every time we switch context. It solves the problem."
But then there's the performance hit. "Flushing buffers takes time. Every single one of these mitigations, to solve hardware bugs, slows down your machine."
The extent of the slowdown depends on the workload. If it is IO-bound, you may hardly notice. But Kroah-Hartman builds kernels. "I see a slowdown of about 20 per cent. That's real. As kernel developers we fight for a 1 per cent, 2 per cent speed increase. Put these security things in, and we go back like a year in performance. It's sad."
Kroah-Hartman dispelled the idea that an issue like Spectre has a single fix. "We are still fixing Spectre 1.0 issues [almost] two years later. It's taken a couple of thousand patches over [almost] two years. Always take the latest kernel and always take the latest BIOS update."
The CVE database of security issues is irrelevant when it comes to the Linux kernel, he said. "CVEs mean nothing, for the kernel. Very few CVEs ever get assigned for the kernel. I'm fixing 20 patches a day, I could create a CVE to each one of them, I was told not to because it would burn the world down," he said.
"If you're not using a supported distro, or a stable long-term kernel, you have an insecure system. It's that simple. All those embedded devices out there, that are not updated, totally easy to break. If you are running in a secure environment and you trust your applications and you trust your users then get the speed back. Otherwise, running in a shared environment, running untrusted code, you need to be secure."
Is AMD safer than Intel? "All the issues that came out this year, were reported not to be an issue on AMD," he told us. Would he enable SMT on AMD? "As of today, that is still a safe option from everything I know. Yes."
Are the MDS vulnerabilities being actively exploited by malware? "They're not that hard to exploit," Kroah-Hartman said. "The research has proved how to do it. The hard part is, you can't tell if somebody is exploiting it. But it is a known problem, you can reproduce it yourself. The Zombieland guys have a great demo. It's a real issue, you need to fix it."
As it happens, next up for The Register at the Open Source Summit was SUSE EMEA CTO Gerald Pfeifer. Naturally, we asked him whether SUSE ships with hyper-threading on or off by default.
"On," he said. "Greg K-H? He's right. Ultimately every customer needs to decide, because there is a cost associated with it. But from a technical perspective he's right."
Imagine, he said, you were Google. "Making that switch means one, two, three more data centres. I'm not arguing leave it on. All I'm saying is, it's not an easy choice. Because someone is going to yell at you if something takes longer."
So there you have it. If you're running on Intel, but want to be secure: best practice is to disable hyper-threading and keep your BIOS and kernel up to date. In reality, though, many factors conspire against that best practice being achieved. ®