This article is more than 1 year old
If malware wants to bury deep inside your Lenovo or Gigabyte servers, they can just ask Vertiv's insecure BMC firmware
Software nasties, hackers, rogue admins can exploit bug duo to hide in your systems
Updated A pair of vulnerabilities in BMC firmware used in servers built by Lenovo – and in Acer and Penguin Computing boxes using Gigabyte server motherboards – can be exploited to hide malware deep below the operating system, hypervisor, and antivirus.
Said spyware could lurk out of sight and undetected by the OS, security tools, administrators, and users; could potentially snoop on and tamper with software and data on the machine; and persist in the motherboard's flash chips allowing it to survive wipes or replacements of storage drives.
Researchers at US-based Eclypsium on Tuesday detailed the flaws: they can be exploited by a malicious logged-in user, or malware, with admin credentials to inject surveillanceware into the system's firmware.
Obviously, if someone or something awful has admin or root-level access to your server kit, it's already pretty much game over for you, though that's not the point here: these bugs can be abused by someone who has already owned your kit to bury deeper, potentially avoiding detection and persisting across reboots and drive wipes.
To hear Eclypsium tell it, the vulnerabilities specifically lie within Vertiv's MergePoint EMS baseboard management controller (BMC) firmware, which is sold to Lenovo for use in its server products, and Gigabyte to use in motherboards sold to Acer, Ciara, Penguin Computing, sysGen, Bigtera, and AMAX, again for use in server products.
The flaws were discovered in Vertiv's code by Eclypsium, reported to Lenovo in July 2018, and first patched in November that year. The holes were found again in BMC firmware on Gigabyte server mobos in March of this year, and traced back to Vertiv, formerly known as Avocent, in April. Now, with the vendors having had ample time to address the coding blunders, Eclypsium is going public.
The BMC in OpenBMC stands for 'Burglarize My Computer' – thanks to irritating security flawREAD MORE
BMC chips are kinda like a super-janitor on your server; a law unto themselves. They sit separate from the main processors and peripherals though have full access to the hardware, and run their own firmware, such as Vertiv's, on their built-in CPU cores. They are typically accessed over the network or via fixed serial cables by data center admins to remotely monitor, manage, and reboot boxes. They can be told to reinstall or repair a node's operating system. They are extremely handy if you need to look after more than a trivial number of machines as they save you the hassle of physically visiting each box one by one.
Eclypsium's crew found two issues in Vertiv's MergePoint EMS technology. One is that it does not bother to perform cryptographic signature checks on its firmware updates before they are installed in the BMC flash storage, meaning non-approved updates or firmware that has been tampered with could be flashed onto the board by privileged malware or users. That would allow an attacker to give themselves persistent and full control over the infected box, or even brick the BMC, by installing malicious BMC firmware.
Second, there is a shell-command-injection vulnerability, designated CVE-2018-9086, in Vertiv's firmware. If you don't fancy crafting malicious BMC firmware images to install via the first vulnerability, CVE-2018-9086 will let you inject shell commands direct into the Linux environment running on the BMC via its update mechanism, and alter its software and scripts that way.
"Malicious modifications to the BMC firmware can be used by an attacker to maintain persistence in the system and survive common incident response steps such as reinstallation of the operating system," Eclypsium noted.
"Additionally, an attack could modify the environment within the BMC to prevent any further firmware updates through software mechanisms, thus enabling an attacker to 'brick' (permanently disable) the BMC through software means."
Reactions to the disclosure have been mixed.
Lenovo said the command-injection flaw was present in five of its ThinkServer lines: the RD340, RD440, RD540, RD640, and TD340. Fixes for all but the RD540 were released in November; the RD540 was patched in April this year. So if you're using Lenovo server lines, and you're up to date with system firmware, you should be fine. If not, check to see if you're affected by CVE-2018-9086, and install updates as needed if you're able to.
Bear in mind, the 2014-era BMC in Lenovo's servers cannot perform cryptographic firmware verification, due to their design, so this issue will remain unpatched on that hardware; later generations of Lenovo's System x and ThinkSystem servers can perform firmware signature checks, though.
It's understood Gigabyte quietly released a fix for the command-injection hole in its Vertiv-based AST 2500 BMC on its server motherboards in May 2019. There's no word on whether cryptographic checks for firmware images have been implemented. Gigabyte's AST 2400 remains unfixed, too, we're told. If you're using a Gigabyte motherboard in your server fleet, check the state of the BMCs if you're worried about these vulnerabilities.
Vertiv, based in Ohio, meanwhile, has not responded to attempts to alert it to its insecure code. Gigabyte couldn't be reached for comment. ®
Updated to add
"As a leading provider of BMC firmware to the OEM community, Avocent began working with key customers as early as 2012, before it was common in the industry, to encrypt and provide verification that the software or firmware being updated was from a trusted source," a Vertiv spokesperson told The Register,
"In 2014, Avocent released a feature upgrade for the MergePoint EMS BMC firmware platform that included verification signing. During the past year, we were alerted to the command line concern and quickly developed and released a patch for our customers."
"We are not aware of any issues related to this, and it’s important to note that the issue identified by the researcher could not have been used to penetrate a network or system. Only someone with access to the system could exploit it."
"We appreciate researchers bringing matters like this to our attention. It helps strengthen our products, and provides an opportunity to remind all consumers and businesses to regularly install software updates and patches to keep their systems current.”