Many free-of-charge antivirus products fail to protect Android smartphone against malware effectively, leaving users with a false sense of security as a result.
Tests by antivirus testing lab AV-Test.org revealed that the best freebie Android anti-virus scanner, Zoner Antivirus, caught 32 per cent of 160 recent Android threats. The other six free-of-charge Android products fared abysmally, with the best of the rest detecting just 10 per cent of the threats. One detected none whatsoever.
AV-Test.org tested seven free-of-charge anti-virus products that it downloaded from the Android marketplace, after searching "anti-virus". The most widely used of these – Antivirus Free from Creative Apps – has over a million users but is still way behind either Lookout Mobile Security and AVG's DroidSecurity, which number 12 million and 10 million plus users respectively. AV-Test.org omitted these products from the tests because Lookout also offers a paid-for security software for Android and, in the case of DroidSecurity, because the technology was recently acquired by AVG (and rechristened AVG Mobilation).
The omission of the products from the tests mean that AV-Test.org's test results are less than comprehensive. But even their findings of a less than complete sample of Android anti-malware products are a real eye-opener, not least because they come from one of the few recognised authorities in anti-virus testing.
Each of the tested security software products was installed on an Android smartphone deliberately infected with inactive specimens of more than 150 recent Android threats. AV-Test.org ran on-demand scans in each case, recording how many threats were detected.
AV-Test.org also included test on F-Secure Mobile Security and Kaspersky Mobile Security, both commercial products, for comparison purposes. Kaspersky and F-Secure both detected more than 50 per cent of threats analysed, substantially better than any of the freebie products tested though poor when compared to the performance of their desktop products.
The second half of these tests involved deliberately attempting to infected freshly cleaned devices with 10 strains of Android malware. Products from F-Secure and Kaspersky detected and blocked all the samples. Zoner Antivirus blocked eight while the other six freebie products blocked either one or none. BluePoint AntiVirus Free, Kinetoo Malware Scan and Privateer Lite warned against one malicious app. Antivirus Free by Creative Apps, GuardX Antivirus and LabMSF Antivirus beta failed completely.
Paid-for apps beat freebies
"In general, the free products didn't perform very well (with just one exception), but the commercial products which were tested as reference performed significantly better," Andreas Marx, chief executive officer of AV-Test.org, told El Reg. "We're working on a review with a focus on commercial apps within the coming weeks."
Marx explained the rationale for the omission of both Lookout and DroidSecurity from this round of tests.
"The product selection is based on the criterion of how common the different freeware anti-virus products are (including their user ratings), based on the Android market scores/data. We wanted to limit the testing to no more than 10 products total in order to perform everything in a timely manner," Marx told El Reg.
"In this first Android test-run, we focused on 'free' anti-virus offerings (the two commercial products from Kaspersky and F-Secure were included as reference only with no final scores given). We consider Lookout's offering as a commercial product, despite the fact that there is also a freeware edition available. The product also includes much more features than a dedicated anti-virus offering. Other products like 'DroidSecurity' were not included, as this one was recently acquired be AVG Technologies, so we considered it also as 'commercial' product."
A greater range of Android security products will be put through their paces in further tests by AV-Test.org.
"As we have received an enormous feedback on this first Android security test report, we will perform further Android reviews in near future which are focusing on much more Android security products and anti-virus offerings. This one will include 'freeware' and 'commercial' offerings from a wide range of vendors," he added.
AV-Test.org's full report on anti-virus scanners for Android can be found here [PDF].
The scanning test set contained 83 Android installation packages (APK) and 89 Dalvik binaries (DEX).
Sean Sullivan, security advisor at F-Secure, explained that its Android security software deliberately avoids detecting binaries because they can lead to false positives.
Because of this the scanning results might be misleading, he said, adding that F-Secure's security caught all the tested malware variants when they actually tried to execute.
Despite making this point, Sullivan described AV-Test's methodology as "fair enough" because it tested in the same way for every product evaluated. ®
Updated to add
Maik Morgenstern Of AV-Test has got in touch to explain that its latest run of tests were more comprehensive then last year's batch because they offered a comparative review of the effectiveness of Droid security scanners.
The most recent tests accessed the effectiveness of 41 Android anti-virus scanners in detecting 618 samples, from 20 different important families of Android malware.
"Last year's test included free products only from more or less unknown vendors (at least no vendor known from traditional desktop IT)," he said.
"The point was to show, that even if there are free products that perform well (like Zoner) you should be careful when selecting your security solution, since there are products that effectively don't do anything at all but display ads. So this was not exactly a comparative review."
Morgenstern added: "This time we were trying to cover as many Android security apps as we could find. No matter whether they are free or paid versions, from known or unknown vendors. The goal of the report is to give an overview of the current malware detection rates of the products. So this one can be considered a comparative review (but it only covers detection rates, none of the other security features)."