Software that claims to detect the presence of OpenSSL's Heartbleed bug in servers, PCs and other gear may falsely report a system to be safe when users are actually in danger, according to a security consultancy.
This finding is disputed by developers publishing tools that test for the vulnerability.
The teams behind Nessus, Metasploit, Nmap and others have each released utilities for sensing whether or not computers and gadgets are affected by the password-leaking Heartbleed flaw. "The problem is, most of them have bugs themselves which lead to false negatives results: that is, a result which says a system is not vulnerable when in reality it is," claimed Adrian Hayter, senior penetration tester at security consultancy CNS Hut3.
"With many people likely running detection scripts or other scans against hosts to check if they need to be patched, it is important that these bugs be addressed before too many people develop a false sense of security regarding their infrastructure," he added.
Hayter has put together a list of tools and scripts that he claims are faulty in a blog post here. Hayter said most of the tools available failed to detect the Heartbleed vulnerability on the Hut3 proof-of-concept server.
The results provide evidence that while the scripts are useful for demonstrating vulnerabilities, they should not be used as a tool for confirming whether systems are secure or not, according to CNS Hut3.
Both Rapid7, which markets Metasploit, and Tenable Network Security (Nessus) said they had modified their security testing technology in response to CNS Hut3's research – although they nonetheless questioned the security consultancy's methodology. Each vendor defended the general effectiveness of its Heartbleed probe.
OK - but you wouldn't see that setup in the wild...
Renaud Deraison, chief research officer at Tenable Network Security and the author of Nessus, said the firm had modified its technology in response to CNS Hut3's research, even though he questioned its methods. “The setup outlined in the April 14 blog in CNS Hut3 blog is interesting because it narrows down TLS so much that most web clients would not be able to connect to a server configured that way," Deraison explained.
"While our original check failed at negotiating this particular cipher, we've since modified it to support more cases like this one. There are many other ways where a check could fail however, for instance a lot of the public proof-of-concepts only test https, but completely ignore other services using SSL such as SMTP, IMAP or OpenVPN.
"Our research team has been working around the clock to cover as many of these services as possible since day one, and we're continuing to investigate other programs using SSL in a non-standard way,” he added.
Tenable has since refined the plugin that CNS Hut3 had deemed faulty: it now detects what the security vendor described as an "edge case". The security vendor has almost 20 Heartbleed detection plugins for local and remote checking with Nessus and SecurityCenter, and also provides detection via passive and log analysis.
Rapid7 told El Reg that it had four free testing tools for Heartbleed and that these were "steadily updated & improved as info or bugs are reported".
Achy breaky heart
CNS Hut3 said it didn’t encounter any false positives: it only saw incorrect negative readings when it put the testing tools through their paces.
Unsurprisingly, the penetration-testing firm has also developed its own standalone tool for detecting whether systems are affected by the Heartbleed OpenSSL vulnerability, which it tested alongside established pen testing utilities such as Metasploit.
It said one sysadmin reported that CNS Hut3's script made HP iLO servers unresponsive.
Hayter said the incident was isolated but it does illustrated the importance of quality assurance in safely testing for Heartbleed (and many other) vulnerabilities.
"There are always dangers with vulnerability testing, because ultimately to test for these vulnerabilities you have to try to exploit them, and whilst you can write exploits that safely work on 99.99 per cent of systems, there’s always going to be that 0.01 per cent which react differently," Hayter told El Reg. "The problem we have here is that Heartbleed is such a dangerous bug, and people want to know immediately if they are vulnerable, so waiting for QA processes to complete before testing is not an option."
He added: "There is a great way to test for this vulnerability without running scripts at your systems: check the version of OpenSSL installed. Of course, whilst this can be done by organisations with small number of machines, it will be a big task for the larger companies, especially if they didn’t have a patching policy in place that covered Linux systems."
Heartbleed is a bug in a cryptographic library that ships with OpenSSL, uncovered last week but present for two years, that creates a means to lift sensitive data such as cryptographic keys from the memory of systems.
Heartbleed exploits work by sending a TLS heartbeat request with a certain number of bytes as a code (eg, the word “CNS”, which is three bytes in UTF-8) but telling the server that the code is actually longer. The server performs no check that the requested code is the length claimed by the request, so it responds with both the code and the specified number of extraneous byres stored after the code in the server memory.
All a detection script has to do is check whether the response code from the server is longer than the code that was sent. "False positives are actually quite hard to come across because of the way Heartbleed is detected," according to Hayter. ®