Security vendors cry foul over exploit tests

'Like testing ABS brakes by pushing a car over a cliff'


Anti-malware vendors have launched a counter-attack on a study questioning the effectiveness of internet security suites, suggesting that the methodology in tests carried out by vulnerability notification firm Secunia was fundamentally flawed.

As previously reported, Secunia tested a selection of 12 internet security suites against how well they did at blocking exploits. None came out of this particular well, with the highest scorer, Symantec, thwarting only 64 out of 300 exploits.

Firms whose products featured in the tests, including Panda Security, cried foul, saying the tests only looked at one of a battery of defensive measures their suites offer. Independent testing organisation AV-test.org backed this line in criticising the tests as focusing purely of on-demand scanning of potentially malicious files. Meanwhile, security firms not involved in the Secunia's bake-off, such as Sunbelt Software, also waded in to cry foul, decrying the exercise as a publicity stunt.

Thomas Kristensen, chief technology officer at Secunia, responded to this criticism by saying vendors had misunderstood the purpose of the tests. While acknowledging its tests weren't comprehensive, Kristensen argued that they illustrated the importance of patching vulnerable applications and adopting a "defense in depth" approach to defending against hackers. He said users shouldn't be lulled into the belief that simply installing and updating internet security suites was enough.

"We only tested one specific aspect (exploitation of vulnerabilities) because too many users believe (and are lead to believe by the marketing material) that they only need a security suite to protect them against various threats including hackers," Kristensen told El Reg.

"Our point is not that Internet Security Suites are useless (they are quite useful for most users). Instead, our point is that they protect insufficiently against hackers and that it is better to prevent attacks by patching rather than relying on other security measures alone".

Panda Security virus analysts Pedro Bustamante compares the Secunia tests to testing a car’s ABS breaking systems by "throwing it down a 200 meter cliff" in a passionate, but nonetheless technically illuminating, blog posting.

"If you only test one part of a product against exploits, which by the way is the part of the product which is not designed to deal with exploits, and leave out of the test the part of the product that DOES deal with exploits and vulnerabilities, there's a very good chance the results will be misleading," Bustamante writes.

"Internet Security Suites do not rely on signature detection alone since many years ago. Panda's (and other) products integrate behavioral analysis, context-based heuristics, security policies, vulnerability detection, etc. However none of these technologies were tested by Secunia."

Bustamante says a number of exploits listed as not detected by Panda are actually blocked if any attempt is made to run them.

Kristensen responded: "It seems quite odd that the AV-vendors are so busy claiming that they can detect literally anything malicious when executed. If they can do that, why do they then have to push "signature" updates to their software so frequently?

"It is obviously much better to be able to detect malicious content while it is passive instead of relying on (hopefully) being able to catch it once executed," he added.

Secunia has taken some of the substantive points made by Panda on board while defending itself against suggestions that its test might have been unfair. "We find the criticism from Panda useful and if we do conduct another test of the file-based test cases, then we will categorise their performance into: Unzipping, manual scan, and opening of test case with vulnerable application," Kristensen said.

Although Secunia and security vendors are at loggerheads over the implications of the tests, there's general agreement that patching is a key element in keeping systems secure - a point that, if nothing else, Secunia's tests have amply illustrated. ®

Broader topics


Other stories you might like

  • If you're using older, vulnerable Cisco small biz routers, throw them out
    Severe security flaw won't be fixed – as patches released this week for other bugs

    If you thought you were over the hump with Patch Tuesday then perhaps think again: Cisco has just released fixes for a bunch of flaws, two of which are not great.

    First on the priority list should be a critical vulnerability in its enterprise security appliances, and the second concerns another critical bug in some of its outdated small business routers that it's not going to fix. In other words, junk your kit or somehow mitigate the risk.

    Both of these received a CVSS score of 9.8 out of 10 in severity. The IT giant urged customers to patch affected security appliances ASAP if possible, and upgrade to newer hardware if you're still using an end-of-life, buggy router. We note that miscreants aren't actively exploiting either of these vulnerabilities — yet.

    Continue reading
  • Now Windows Follina zero-day exploited to infect PCs with Qbot
    Data-stealing malware also paired with Black Basta ransomware gang

    Miscreants are reportedly exploiting the recently disclosed critical Windows Follina zero-day flaw to infect PCs with Qbot, thus aggressively expanding their reach.

    The bot's operators are also working with the Black Basta gang to spread ransomware in yet another partnership in the underground world of cyber-crime, it is claimed.

    This combination of Follina exploitation and its use to extort organizations makes the malware an even larger threat for enterprises. Qbot started off as a software nasty that raided people's online bank accounts, and evolved to snoop on user keystrokes and steal sensitive information from machines. It can also deliver other malware payloads, such as backdoors and ransomware, onto infected Windows systems, and forms a remote-controllable botnet.

    Continue reading
  • To cut off all nearby phones with these Chinese chips, this is the bug to exploit
    Android patches incoming for NAS-ty memory overwrite flaw

    A critical flaw in the LTE firmware of the fourth-largest smartphone chip biz in the world could be exploited over the air to block people's communications and deny services.

    The vulnerability in the baseband – or radio modem – of UNISOC's chipset was found by folks at Check Point Research who were looking for ways the silicon could be used to remotely attack devices. It turns out the flaw doesn't just apply to lower-end smartphones but some smart TVs, too.

    Check Point found attackers could transmit a specially designed radio packet to a nearby device to crash the firmware, ending that equipment's cellular connectivity, at least, presumably until it's rebooted. This would be achieved by broadcasting non-access stratum (NAS) messages over the air that when picked up and processed by UNISOC's firmware would end in a heap memory overwrite.

    Continue reading

Biting the hand that feeds IT © 1998–2022