Is AV product testing corrupt?

Who can you trust?

Let's take this one step further, let's say that I put in a cancer sample that isn't even a cancer. It's a benign form of a cell structure. In the previous analogy then I could be told I have cancer when in fact I don't. I could be forced to undergo surgery just because of a faulty detection.

Now, one step even further. Now I submit a "cancer" sample that is ONLY detected by my drug company. The implications become quite obvious in this analogy. Why aren't they obvious to the users of today's anti-virus products? Because there are only a couple of companies out there that are in the business to "sell" their "so called" unbiased review of the antivirus products on the market.

In a major industry publication, the following was quoted from one of these "anti-virus reviewers", they indicated, "by his own admissions [he] does not verify that everything submitted to his list is in fact malware." Taking "just" that fact alone with nothing else, should lead any person to question the tests as a good measure of an anti-virus product's effectiveness in preventing a computer infection.

If this whole comparison testing wasn't complicated enough, then let's add to it the changes in the anti-malware industry that are adding new features and functionality to the products. Some of the anti-virus companies have added anti-spyware capabilities, some anti-spyware companies have added anti-virus scanning to their products, and some "security suites" are including other kinds of security protection including firewall and even host-based intrusion prevention.

Are any of these blocking techniques included in the evaluation or detection comparisons? No. Isn't it more important how secure the product keeps the user rather then just simply how many samples from a tainted, untested, unvalidated, out-of-date and highly dubious boxed set are detected?

The public is getting a VERY corrupt and biased view that doesn't relate even remotely to the real world level of malware protection. This type of testing is very biased and yet the public is generally unaware of the faith that they place in several magazines that are promoting the results of these fly-by-night companies with lack of industry standard credentials.

In the end, the computer user doesn't really care about the rate of detection or the features, they just want their computer to be protected as best as they can possibly make it with administration that's as easy as possible. My computer hasn't had a malware infection in YEARS and according to the latest "review" in a major magazine, my product was in the bottom half. Hmmm, kind of makes you wonder, doesn't it?

The anti-malware industry needs a "gold standard" to guide the development of a fair and truly unbiased measure of the product's effectiveness. This gold standard needs to be untainted by graft, ignorance and pseudo-science. The public deserves it. The industry should pull together to force it. Until then, place your trust with anti-malware companies with a proven history of performance... or, just hope you don't get hit using one of the reviewer's "best" anti-virus products.

Copyright © 2007, IT-Analysis.com