Windows Defender spyware-blocking under fire (again)

Who tests the testers?


Microsoft's Windows Defender has once again come under criticism for alleged shortcomings in blocking invasive spyware applications.

In tests sponsored by anti-spyware vendor PC Tools, and carried out by independent testing facility Enex Testlabs, Microsoft’s Windows Defender blocked less than half (46 per cent) of current spyware threats, scoring well below third party anti-spyware providers.

The findings, published on Tuesday, follow earlier in-house research by security rival Webroot that Windows Defender failed to block 84 per cent of a testing sample-set that included "15 of the most common variations of existing spyware and malware". Threats of various types - including adware, system monitors, key loggers and Trojans - were able to reside on the testing environment undetected by Windows Vista, Webroot reports.

Microsoft has declined to comment on Webroot's criticism and is also staying quiet on the latest reports.

Microsoft's rivals are understandably keen to promote the message that while Vista might be more secure than previous versions of Windows, users still need additional protection from malware threats. Redmond itself isn't up to the job so users ought to continue relying (buying) third-party products, the argument goes.

The problem with both the PC Tools and Webroot's survey is the result of the tests depends on the spyware sample used, who supplies it and the complete objectivity of the testing agency. In the case of the Webroot test, the sample data was "randomly chosen from a database of over 8,000 spyware installation programs that was provided by Webroot" (our emphasis). PC Tools criticised this approach as "hand picking" the sample set. It said PC Tools did not choose or supply the sample-set used by Enex, described as real-world spyware threats circulating in 2006.

According to the aggregate Enex test results for the whole of 2006, Microsoft’s Windows Defender quick scan was able to block only 47 per cent of dangerous threats while their full scan blocked 53 per cent. Tested at the same time and using the same sample-set, PC Tools’ Spyware Doctor quick scan blocked 83 per cent and the full scan blocked 89 per cent better than other unnamed anti-spyware products put through their paces.

However since PC Tools hired Enex to conduct the tests they inevitably carry less authority than would be the case if they were done completely independently.

In the anti-virus world, vendors have agreed to submit to testing against a set of viruses at large on the internet in tests conducted by independent testing houses such as Virus Bulletin and not paid for by any one vendor. The anti-spyware industry hasn't reached this level of maturity.

It's quite possible, for example, that Webroot and PC Tools products might only detect a small proportion of a sample set supplied by Microsoft or McAfee or anyone else. That's not to say these products are ineffective, simply that tests are meaningless until an independently-produced sample is used in tests conducted by wholly disinterested parties.

Until we get to that point anti-spyware tests will be more about marketing than objective product performance assessments. ®

Similar topics


Other stories you might like

  • It's primed and full of fuel, the James Webb Space Telescope is ready to be packed up prior to launch

    Fingers crossed the telescope will finally take to space on 22 December

    Engineers have finished pumping the James Webb Space Telescope with fuel, and are now preparing to carefully place the folded instrument inside the top of a rocket, expected to blast off later this month.

    “Propellant tanks were filled separately with 79.5 [liters] of dinitrogen tetroxide oxidiser and 159 [liters of] hydrazine,” the European Space Agency confirmed on Monday. “Oxidiser improves the burn efficiency of the hydrazine fuel.” The fuelling process took ten days and finished on 3 December.

    All eyes are on the JWST as it enters the last leg of its journey to space; astronomers have been waiting for this moment since development for the world’s largest space telescope began in 1996.

    Continue reading
  • China to upgrade mainstream RISC-V chips every six months

    Home-baked silicon is the way forward

    China is gut punching Moore's Law and the roughly one-year cadence for major chip releases adopted by the Intel, AMD, Nvidia and others.

    The government-backed Chinese Academy of Sciences, which is developing open-source RISC-V performance processor, says it will release major design upgrades every six months. CAS is hoping that the accelerated release of chip designs will build up momentum and support for its open-source project.

    RISC-V is based on an open-source instruction architecture, and is royalty free, meaning companies can adopt designs without paying licensing fees.

    Continue reading
  • The SEC is investigating whistleblower claims that Tesla was reckless as its solar panels go up in smoke

    Tens of thousands of homeowners and hundreds of businesses were at risk, lawsuit claims

    The Securities and Exchange Commission has launched an investigation into whether Tesla failed to tell investors and customers about the fire risks of its faulty solar panels.

    Whistleblower and ex-employee, Steven Henkes, accused the company of flouting safety issues in a complaint with the SEC in 2019. He filed a freedom of information request to regulators and asked to see records relating to the case in September, earlier this year. An SEC official declined to hand over documents, and confirmed its probe into the company is still in progress.

    “We have confirmed with Division of Enforcement staff that the investigation from which you seek records is still active and ongoing," a letter from the SEC said in a reply to Henkes’ request, according to Reuters. Active SEC complaints and investigations are typically confidential. “The SEC does not comment on the existence or nonexistence of a possible investigation,” a spokesperson from the regulatory agency told The Register.

    Continue reading

Biting the hand that feeds IT © 1998–2021