Google questions tests that praise IE's bad website blocker

Er, they're testing against Chrome 6

The methodology of tests that found IE is tops for blocking a particular type of malware attack have come under fire from Google.

NSS Labs was commissioned by Microsoft to access the ability of browser to block socially engineered malware attack URLs. The exercise focused on the effectiveness of in-built browser technology to block access to malware portals or sites known to be contaminated with malicious code (e.g. scareware portals). Such protection should be supplemented by anti-virus protection, as NSS makes clear.

The tests found that Internet Explorer 9 beta had a 99 per cent detection rate of malicious URLs, IE 8 hit a creditable 90 per cent, with Firefox 3.6 on 19 per cent, Safari 5 hit 11 per cent in the benchmark with Google's Chrome 6 browser achieving just three pre cent. Standalone Opera offered no protection from this type of attack, according to the tests.

Microsoft commissioned the results as a private benchmark, authorising their publication on the receipt of favourable results. That's not the only reason to question whether the tests should be read at face value. The exercise was conducted with Chrome 6 even though the latest version of the open sauce browser is Chrome 8. In a statement, Google defended the overall security of its browser and questioned the value of NSS's tests.

These sponsored tests are limited in their sole focus on socially engineered malware, while excluding vulnerabilities in plug-ins or browsers themselves. Additionally, the testing methodology isn't available in a way that can be independently verified. Google Chrome was built with security in mind from the beginning and emphasizes protection of users from drive-by downloads and plug-in vulnerabilities — for example, we recently introduced a new security sandbox for Flash Player.

Security testing is often a contentious business, with even the methodology of unpaid, independent tests from the likes of Virus Bulletin coming into question from time to time. That observation goes double for sponsored tests, especially those that focus on a narrow security metric, however competent the actual testers, and NSS has some of the best in the industry. ®