Anti-spam packages ‘too unreliable’ to certify

Testing. Very Testing


Poor performance of anti-spam packages is frustrating attempts by a leading certification vendor to develop benchmarking standards.

ICSA Labs, which runs one of the most important security industry certifications programmes, has recorded disappointing results in its preliminary tests of eight open source and commercial anti-spam packages.

Larry Bridwell, programme manager at ICSA Labs, an independent division within TruSecure, said it had "trouble getting up to 60 per cent" recognition of spam email in the products it has tested thus far.

ICSA Labs is well known for testing a wide variety of security products - anti-virus, cryptography, PKI, IPSec, VPN, firewall, intrusion detection and content filtering - against specific criteria for compliance and reliability. Products which pass the test - and most do not at the first attempt - earn the right to bear the ICSA Labs Certification Seal. Further testing is carried out to make sure products continue to meet certification standards.

Plans to create a certification programme for anti-spam products flowed out of ICSA's work in certifying content filtering packages.

George Japak, VP of ICSA Labs, told us that in the "near term there's unlikely to be a formal certification programme" for anti-spam products.

"It's hard to say 60 to 70 per cent is good enough," he said.

However stakeholders in ICSA certification programme - including developers, corporate end-users and independent researchers - are arguing that even 60 per cent efficiency contributes to reducing the spam problem.

In our own use of anti-spam packages like SpamPal and MailWasher, we found it fairly straightforward to achieve (we estimate) efficiency levels for blocking spam of 80 per cent or above. However this has at the expense of occasional (less than one in 100) false positives.

There is a disadvantage in using these products of mail downloads slowing to a crawl but the overall benefits of these packages outweigh their pitfalls. We also hear good things about SpamAssassin.

Doubtless other products and services achieve better results but without comprehensive independent testing it's hard to be sure what package to use.

The recent proliferation of anti-spam products and difficulties in agreeing testing criteria are complicating factors.. In testing only eight products, ICSA Labs is looking at only a very small sample of the scores of packages out there - a point that that hasn't gone unnoticed at ICSA.

We shouldn't read too much into ICSA Labs preliminary results, but they do point to the immaturity of the anti-spam market.

Longer term the future looks much brighter. With the best minds in the IETF, and elsewhere, focusing on the problem, gives reason for optimism.

Current research efforts to apply Bayesian analysis in weeding out spam messages from legitimate email and to slow spammers down using tar pits appear most promising. ®

Related stories

On spam cures that are worse than the disease
Web giants to declare war on spam
Florida spammers sue anti-spam groups
Rise of the Spam Zombies
IETF aims to can spam
Spammers break law with covert tracking
Plaid up in arms as Commons spam filter bans Welsh
Vendors complete tougher ICSA 4.0 firewall tests


Other stories you might like

  • GPL legal battle: Vizio told by judge it will have to answer breach-of-contract claims
    Fine-print crucially deemed contractual agreement as well as copyright license in smartTV source-code case

    The Software Freedom Conservancy (SFC) has won a significant legal victory in its ongoing effort to force Vizio to publish the source code of its SmartCast TV software, which is said to contain GPLv2 and LGPLv2.1 copyleft-licensed components.

    SFC sued Vizio, claiming it was in breach of contract by failing to obey the terms of the GPLv2 and LGPLv2.1 licenses that require source code to be made public when certain conditions are met, and sought declaratory relief on behalf of Vizio TV owners. SFC wanted its breach-of-contract arguments to be heard by the Orange County Superior Court in California, though Vizio kicked the matter up to the district court level in central California where it hoped to avoid the contract issue and defend its corner using just federal copyright law.

    On Friday, Federal District Judge Josephine Staton sided with SFC and granted its motion to send its lawsuit back to superior court. To do so, Judge Staton had to decide whether or not the federal Copyright Act preempted the SFC's breach-of-contract allegations; in the end, she decided it didn't.

    Continue reading
  • US brings first-of-its-kind criminal charges of Bitcoin-based sanctions-busting
    Citizen allegedly moved $10m-plus in BTC into banned nation

    US prosecutors have accused an American citizen of illegally funneling more than $10 million in Bitcoin into an economically sanctioned country.

    It's said the resulting criminal charges of sanctions busting through the use of cryptocurrency are the first of their kind to be brought in the US.

    Under the United States' International Emergency Economic Powers Act (IEEA), it is illegal for a citizen or institution within the US to transfer funds, directly or indirectly, to a sanctioned country, such as Iran, Cuba, North Korea, or Russia. If there is evidence the IEEA was willfully violated, a criminal case should follow. If an individual or financial exchange was unwittingly involved in evading sanctions, they may be subject to civil action. 

    Continue reading
  • Meta hires network chip guru from Intel: What does this mean for future silicon?
    Why be a customer when you can develop your own custom semiconductors

    Analysis Here's something that should raise eyebrows in the datacenter world: Facebook parent company Meta has hired a veteran networking chip engineer from Intel to lead silicon design efforts in the internet giant's infrastructure hardware engineering group.

    Jon Dama started as director of silicon in May for Meta's infrastructure hardware group, a role that has him "responsible for several design teams innovating the datacenter for scale," according to his LinkedIn profile. In a blurb, Dama indicated that a team is already in place at Meta, and he hopes to "scale the next several doublings of data processing" with them.

    Though we couldn't confirm it, we think it's likely that Dama is reporting to Alexis Bjorlin, Meta's vice president of infrastructure hardware who previously worked with Dama when she was general manager of Intel's Connectivity group before serving a two-year stint at Broadcom.

    Continue reading
  • Lithium production needs investment to keep pace with battery demand
    Report says $42b will need to be poured into industry over next decade

    Growing demand for lithium for batteries means the sector will need $42 billion of investment to meet the anticipated level of orders by the end of the decade, according to a report.

    Lithium is used in batteries that power smartphones and laptops, but there is also rising use in electric vehicles which is putting additional pressure on supplies.

    The report, Benchmark Mineral Intelligence, predicts that demand will reach 2.4 million tons of lithium carbonate equivalent by 2030, roughly four times the 600,000 tons of lithium forecast to be produced this year.

    Continue reading
  • Cars in driver-assist mode hit a third of cyclists, all oncoming cars in tests
    Still think we're ready for that autonomous future?

    Autonomous cars may be further away than believed. Testing of three leading systems found they hit a third of cyclists, and failed to avoid any oncoming cars.

    The tests [PDF] performed by the American Automobile Association (AAA) looked at three vehicles: a 2021 Hyundai Santa Fe with Highway Driving Assist; a 2021 Subaru Forester with EyeSight; and a 2020 Tesla Model 3 with Autopilot.

    According to the AAA, all three systems represent the second of five autonomous driving levels, which require drivers to maintain alertness at all times to seize control from the computer when needed. There are no semi-autonomous cars generally available to the public that are able to operate above level two.

    Continue reading

Biting the hand that feeds IT © 1998–2022