Many antispam products still block an inconvenient amount of legitimate e-mail, a new test of leading products has suggested.

Slideshow: Famous last words about spam

The data emerged from preliminary tests carried out by the UK-based outfit Virus Bulletin (VB) using a methodology still being worked on by the company.

Although none of the eight products looked at are mentioned by name in the published results - reckoned to be unfair given the trial nature of the tests - all blocked between 0.4% (the worst performing) and 0.04% (the best performing) of legitimate ham' e-mails.

The qualification is that the length of time the different products ran did vary from only three days to ten days, making comparisons impossible, but none appeared to be able to improve on this modest performance. For perspective, a false positive rate of even 0.04% over seven days of testing was on one product under test was equivalent to blocking 653 legitimate e-mails by VB's definition, or 93 per day.

"The e-mails concerned have been scrutinized and the majority proved to be e-mails from mailing lists and newsletters - e-mail that is sent in bulk and notoriously difficult for filters to distinguish from spam," said the VB testers. "Of course, being difficult for the filters to distinguish is not an excuse for filters to block the mails."

The ability of the products to block genuine e-mails was only slightly more convincing, varying from 89.5% to 95.59%. A product achieving a basic silver' stamp (above which lies Gold and Platinum) would need to achieve a spam-catch rate of 85% or above, with a false positive rate lower than 0.25%.

"We haven't mentioned the product names this time because this was a trial run - vendors submitted their products to us free of charge on the understanding (because the methodology was previously untested) that the published results would be anonymised (the vendors themselves have been supplied with their own test results)," said Virus Bulletin's Editor, Helen martin.

After announcing its intention to create an antispam test methodology in January, VB now says it plans to finalize a full run in April, with the results to be published in May. These will relate vendors to their full scores without the cloak of anonymity for protection.