Junk mail is an increasing problem but the search for an effective cure remains fraught with difficulties.
That's the message we take from a slew of recent surveys on the subject.
Market watcher IDC predicts that anti-spam products will be a key driver for the secure content management (SCM) software market which it expects to grow by 19 per cent a year to reach $6.4 billion in 2007.
"Virus infection is still the main concern regarding corporate security, but other factors, such as spam and regulatory compliance, are increasingly driving the adoption of secure content management technologies," said Brian Burke, research manager for IDC's Security Products service. "Anti-spam will continue to be an important adoption driver in the messaging security market. However, IDC believes it will become a feature of messaging security and not a distinct market."
ISC notes that spam is no longer just a nuisance but quickly becoming a potential legal liability and productivity drain for corporate IT departments and users alike. Spam is also another conduit for unknown viral applications into the corporation, for links to pornographic or objectionable Web sites, and for leaks of sensitive company information. For these reasons spam not only place strain on corporate bandwidth and storage, it creates legal pitfalls too.
Message security, which includes anti-spam products, is the smallest segment of the secure content management software market, but will grow the fastest over the forecast period of 2003-2007, increasing from $236 million in 2002 to $1.1 billion in 2007. IDC predicts Web filtering to be the second-fastest-growing segment of the secure management software market, increasing to $893 million in 2007. The largest segment of this market, anti-virus software, will grow the slowest over the forecast period, reaching $4.4 billion in 2007.
IDC looks at the market in more detail in its recently published Worldwide Secure Content Management Forecast Update and Competitive Vendor Shares report.
Annoying spam false positives
One of the main weapons in the fight against spam is filtering tools. However, the possibility exists that legitimate email will not make it through spam filters thanks to spam false positives.
A report this week from Ferris Research suggested that false positives are costing American businesses $3.5 billion this year. This figures comes from an estimate of the time spent chasing up emails, which have been mistakenly filtered out, or rooting through junk mail folders to find legitimate emails.
These figures seem way too high to us. After all, people normally follow up important email messages with a phone call anyway. Meanwhile, Ferris estimates that spam will cost US businesses over $10 billion in 2003 (again, too high, we feel) so it is calling out for a more intelligent approach to the application of spam filtering technology.
Separately, the Radicati Group recently estimated that most filtering software has a false positive rate of between one and ten per cent, a figure closer in line with our own experience of using anti-spam packages. Hopefully with developments in Bayesian filtering techniques this figure will be greatly reduced.
False negatives pose a problem too...
Meanwhile, a study by censorware outfit SurfControl highlights some of the techniques spammers are adopting to avoid detection. Some of the techniques are fairly well known but bear a recap considering they still often evade detection mechanisms.
The techniques incorporate multiple, sophisticated software tricks buried in HTML code to confuse and avoid detection by traditional content filtering mechanisms, such as dictionaries and statistical analysis. HTML-based spam now accounts for 95 per cent of all spam, and 99 per cent of adult spam now identified and analysed by SurfControl.
These techniques include:
- Hidden Agenda - Most commonly used in porn spam, this technique attempts to fool filters by tricks within the HTML source code of the message. Spammers use ASCII control code to represent letters, random words or phrases, as well as white text on a white background in HTML, within HTML comments or in bogus HTML tags, all invisible to an email recipient. The result of these techniques is to split spam words that make them unreadable by dictionary-based scanning tools.
- Treacherous Tracks - This technique allows spammers to use their Web servers to break down a URL directory structure and add code that can verify a user's e-mail address, track them online and redirect an email user automatically to a specific Web page.
- Dodgy Domains - An increasingly common technique used in HTML-based spam, this subterfuge allows spammers to redirect unsuspecting email recipients to unexpected Web sites, typically using an @ sign in url addresses to confuse people about the domain they are really visiting. This technique is often used in "brand spoofing" spam.
- Random Ramblings - This common technique used by less sophisticated spammers involves inserting long random words or characters in a subject line or body of a message. It is designed to skew statistical filtering.
- Counterfeit Characters - A technique that uses numbers or accented characters to replace standard characters to fool filtering dictionaries unless these include spelling variables. V1agra or M0RTG4GE are two common examples of this trick.
SurfControl said that over the past six months research indicates these frequently used spam techniques have become commonly used by the most threatening and offensive spammers: pornographers and those engaged in 'brand spoofing' in order to steal personal information. The techniques (often used in combination within a single message) capitalise on the naïveté of e-mail users and pose significant legal, security, network and productivity risks for businesses, it adds.
"These deceptive tactics are making it easier than ever for spammers to prosper and harder than ever for technology companies and law enforcement officials to identify and stop them," said Susan Larson, SurfControl's VP for Global Content Operations. "In addition, spammers are using offshore Web hosting services that make them very hard to track and email harvesting services that make it simple to target more people than ever before." ®