Product Guide: Rival solutions smack down spam

Product Guide: Rival solutions smack down spam

Spam is such a significant problem these days that even the US federal government has gotten involved, passing legislation such as the CAN-SPAM Act. Meanwhile, a wide variety of companies has collaborated on an authentication scheme called the Sender ID Framework, aimed at making e-mail fraud even harder to perpetrate. Realistically, though, neither CAN-SPAM nor e-mail-sender authentication is likely to do much to stem the flood of mail. Fortunately, anti-spam products continue to evolve and improve.

Spam is such a significant problem these days that even the US federal government has gotten involved, passing legislation such as the CAN-SPAM Act. Meanwhile, a wide variety of companies has collaborated on an authentication scheme called the Sender ID Framework, aimed at making e-mail fraud even harder to perpetrate. Realistically, though, neither CAN-SPAM nor e-mail-sender authentication is likely to do much to stem the flood of mail. Fortunately, anti-spam products continue to evolve and improve. In my ongoing tests of anti-spam products, I recently looked at four more contenders: Brightmail Anti-Spam 6.0 from Symantec Corp.; IMSS (InterScan Messaging Security Suite) Version 2.8 from Trend Micro Inc.; MailFrontier Enterprise Gateway 3.1 from MailFrontier Inc.; and PureMessage 4.6 from Sophos PLC. They all performed admirably, filtering more than 90 percent of spam, with few false positives.

All are software gateways that can reside on the same system as your mail server or on a separate system. They all offer enterprise-caliber features, including user access to quarantined messages; automatic setup of user access to quarantined messages; different policies by user, group, or domain; optional anti-virus scanning; and useful reporting tools. Pricing for all four is within $1 per user, per year, for anti-spam, anti-virus, and policy filtering.

Beyond the basics, Brightmail offers a simple installation with almost no configuration or tuning required, and little ongoing maintenance is necessary. Trend Micro delivers extensive tuning capabilities that cater to the needs of varying groups of users. MailFrontier provides easy installation and great reporting. Sophos installs on Linux only, and it provides a great deal of flexibility as well as a relatively simple installation.

As for accuracy, Brightmail sets the standard for filtering performance, boasting zero false positives, critical or bulk, and stopping 97.69 percent of spam. With zero critical false positives, 1.26 percent bulk false positives, and 96 percent of spam stopped, Sophos will also keep end-users content. MailFrontier is also quite acceptable, with the spam-stopping score of 97.95 percent, three critical false positives out of 1,711 messages, and a bulk false positive rate of 0.94 percent. IMSS is still well within the acceptable range, with a bulk false positive rate of 0.6 percent, one critical false positive out of 834 messages, and 96.5 percent of spam blocked.

The importance of the false positives rate should not be overlooked; that statistic is arguably more significant than a solution's spam-blocking percentage. Mining the quarantine for false positives, after all, is much more time-consuming than dealing with the few spam messages that slip through the filter.

In my tests, I divided the false positives into two categories: bulk and critical. Stopping some bulk e-mails, such as newsletters, mailing lists, and authorized marketing e-mails from getting through is not the worst thing in the world, and it's generally easily remedied by adding a few senders to the whitelist. Critical false positives are personal e-mails addressed to specific users that get blocked. A high critical false positive rate is the biggest barrier for end-user acceptance of anti-spam filters: If it's too high, they stop trusting the filter or have to spend a lot of time checking quarantined e-mail every day.

Symantec Brightmail Anti-Spam 6.0

Brightmail Anti-Spam 6.0 is the latest in a line of products that has been a top performer in our tests during the past couple of years. Recently acquired by Symantec, Brightmail will offer few surprises to those familiar with previous versions.

The solution installs easily on Windows 2000 and 2003 Servers. It does require IIS for SMTP services and will prompt you to install it if it's not present. It would be nice if it also warned you that using the default installation option for IIS through Windows doesn't install the needed SMTP component. It also installs MySQL Pro and the open source Tomcat application server, which handles quarantined messages and grants end-users access to the quarantine. A license key is also required, as is registration through the Symantec Web site.

Configuration is simple, straightforward, and well-documented. When the initial configuration is complete, there is little else to do. There are controls for the filters available, but given the very high performance of the filters in the default position, it's hard to imagine anyone wanting to mess with them. Further, there are no updates to schedule, as they occur automatically.

Brightmail provides Web access to the quarantine on a per-user basis. Users may also access the quarantine via plug-ins for Microsoft Corp. Outlook and Exchange as well as for Lotus Domino. Users may view quarantined messages, release messages incorrectly identified as spam, report spam that got through, and control their whitelist and blacklist settings. User and group information can be imported from Active Directory or other LDAP directories to speed the setup process.

Administration is also performed via browser, and admins can manage multiple servers across the enterprise from a single console. Policies are manageable by domain, group, or users, with fine granularity for controls as well as permissions for end-user access.

Brightmail now offers non-English support. It detects what language is in use in an e-mail for the top dozen languages (including Chinese, Russian, Japanese, Korean, German, and Italian), and heuristics only run for the applicable language. It can also let through messages written in one specific language or in English and another language.

Also new is the sender authentication feature, which ensures that the apparent e-mail address of a sender is legitimate and filters out messages from fake sender addresses before they even hit the server. Brightmail has done considerable work to optimize this feature. The filter rejects messages using the fastest filters first, thus reducing the load increase. Brightmail estimates that even with sender authentication turned on, overall load increases by less than 3 percent.

The results tell all: nearly 4,000 messages and no false positives, not even newsletters or marketing materials. Brightmail is an enterprise-caliber product with superb performance that didn't need to be tuned at all, and that had almost no ongoing maintenance requirement.

Trend Micro InterScan Messaging Security Suite

IMSS is a full-featured anti-spam, anti-virus, and e-mail policy management suite that runs on Linux, Solaris, or Windows 2000 and 2003 Servers. Installation is relatively simple and can be done remotely if desired. When the product is installed, it must be registered via the Trend Micro Web site, which then e-mails activation keys that must be entered.

Configuration is straightforward, and the Web-based interface is easy to navigate, although after you make all your changes and click the Save button in each field, you must click the easy-to-overlook Apply Now button to update all the configuration changes to the server.

I initially received the out-of-date Version 2.0 of the software, which shipped in late May. Unfortunately, the product's Auto-Update feature updated anti-spam signatures but not the software engine, a problem the company says is fixed in the current Version 2.8. The product's accuracy improved dramatically when I installed the newest edition, stopping 96.5 percent of spam and generating only one critical false positive, resulting in a bulk false positive rate of 0.72 percent.

IMSS allows for highly specific tuning of filters, from lenient to aggressive, in a variety of categories including sexual or racial content, profanity, chain letters, hoaxes, and HTML scripts. Filters can be tweaked for individual users, groups, or domains.

Admins may customize the actions taken when a filter is triggered. In addition to the usual defaults of quarantining, forwarding with an addition to the subject heading, forwarding to a different user account, or deleting what the other programs offer, you can also create custom responses. For instance, you could have all e-mails containing objectionable racial or sexual content automatically forwarded to an HR mailbox and with a warning inserted at the top of the message.

Users may access quarantine via a browser interface or an Exchange plug-in, allowing them to release messages and whitelist or blacklist senders. User and group information can be imported from Active Directory or other LDAP directories to speed the setup of users and groups in IMSS.

Aside from the outdated original version of the software issue, IMSS performed well, and it offers extensive policy management tools and granular management of anti-spam characteristics.

MailFrontierEnterprise Gateway 3.1

MailFrontier had the easiest installation of any of the products I tested and, as does IMSS, requires no additional software. The Windows installer automatically installs Tomcat and Java Runtime Engine, which grant access to quarantine. The installer installs to any Windows 2000 Server or Windows 2003 Server system on which you have administrator rights.

Admins may deploy the product remotely to one or more servers with a single install. Be mindful, however, that if you're setting it up as a gateway and not paying close attention, you could inadvertently install it on the mail server instead of the local system, as I did.

When Enterprise Gateway is installed, you may retrieve users and groups from a Windows NT, Active Directory, Novell NDS, or other LDAP directory. Brightmail and Trend Micro offer the same capability. Individual user access to quarantine requires enabling the LDAP function; there's no provision for creating user tables manually nor for automatically creating log-ins based on e-mail address. After user information is imported into the LDAP server on the MailFrontier system, it is automatically updated. New users added to the directory in Windows have access to quarantine as soon as they are enabled in the e-mail directory system.

Users can access the quarantine via browser or by downloading an Outlook plug-in. They can then release quarantined e-mail or report spam that got through the filter, plus they can add addresses to the whitelist or blacklist from within Outlook or via the browser. Releasing e-mail automatically whitelists the sender, which is a nice feature.

MailFrontier provides lots of control to the admin and better-than-average reporting tools, with an easy-to-use interface for generating reports and a wide variety of predefined reports available.

Policy management is flexible and easier to configure. There are two spam categories, spam and likely spam, each of which can have a separate response. So, for instance, with an addition to the subject line, you could quarantine spam and mark messages that are likely spam. Users could then report any likely spam they received as spam, which would fine-tune the filters. Each of five categories of spam (sexual content, offensive language, get rich quick, gambling, and advertisements), can have separate settings from mild to strong filtering, and the administrator can choose whether to allow users to release each of those types of messages from quarantine.

MailFrontier provides excellent accuracy, stopping 97.95 percent of all spam --but a couple of critical false positives did slip through. Furthermore, it's easy to install and import users from Active Directory. The remote installation feature will appeal to admins at large companies who need multiple servers.

Sophos PureMessage 4.6

PureMessage is the only one of these gateways that doesn't support Windows, installing on AIX, HP-UX, Linux, or Solaris. A Windows version should be available later this year. A year or two ago, this would have meant a much more complex installation. Today, though, even Windows admins who have installed Red Hat Linux a time or two should be comfortable installing PureMessage in front of their Exchange or other e-mail server. Linux shops will be satisfied with a high-performance, easy-to-install package.

After I installed Red Hat 9, the PureMessage documentation clearly showed me where the two additional required libraries, libstdc++ and glibc, were located. The PureMessage installer is simple and easy to run, and includes all other packages (your choice of sendmail or postfix), and either the CDB or PostgreSQL databases for quarantine.

PureMessage can either quarantine spam or modify the subject line to highlight likely spam. The documentation has a nice, clear example of how to set up a filter in Outlook to move marked spam to a junk folder. There is no plug-in for Outlook, however, so users cannot release e-mail directly from the e-mail app. Instead, they must access the browser interface to release mail from quarantine or add to their whitelist or blacklist. If the quarantine is enabled, spam can be released and added to the whitelist simultaneously. If it's not, users may easily access suspected spam from a folder via e-mail program but must then take two manual steps to add false positives to their whitelists or spammers to their blacklists. Controls to allow users to change filter settings are granular and easy to set up.

As with the other products in this test, PureMessage includes anti-virus and policy management tools. These are easy to configure through the browser interface when the server is running. Control of users and groups is easy to delegate, with several levels of access possible, so that basic users and power users can have varying levels of control of the filter settings, whitelists, and blacklists. Although user and group information can be imported as text files, direct imports from Active Directory or LDAP is not supported, adding a small step for large networks. The Windows version of PureMessage supports importing users and groups from Windows directories.

PureMessage is a nicely engineered gateway that will probably appeal more to Linux or Unix shops than to Windows shops, although it's easy enough to install and use that the platform should not be a barrier to entry in Windows-only networks. With excellent performance and management, it will satisfy users in any environment.

Decisions, decisions

All of the products in this test provide 95 percent or better spam filtering, which is certainly adequate. Whereas Brightmail shines with perfect false positive performance without training, the other three approach the same zero-defect level with some tuning.

Pricing is very similar for all four products with anti-spam, anti-virus, and policy management installed. If you already have an anti-virus solution or don't need policy management, pricing will vary for your application. Given that all four products provide good user and group management, strong policy enforcement, impressive reporting tools, and useful Web interfaces, pricing may well be the deciding factor.

There are some specialized situations in which specific products may appeal to some admins. For instance, Brightmail is the only product to supply a plug-in for Notes. But Linux or Solaris shops will not find it as appealing, because it works only on a Windows server. The other three products support Solaris, and both IMSS and PureMessage support Linux.

IT consultant Logan Harbaugh is the author of two books on networking. Contact him at


Dam that spam

Spam-fighting standards and services won't make anti-spam products obsolete any time soon

Spam annoys everyone, even Bill Gates. That's why Microsoft recently proposed a technology it called "Caller ID for e-mail" which is intended to verify that the sender shown in the From: part of an e-mail is the real sender. Several other organizations have come up with their own versions of this idea, and the different standards may become integrated into a single standard called the Sender ID Framework -- if all interested parties can come to an agreement.

In the meantime, there are several incompatible standards, some of which are already in use by large ISPs such as Yahoo. Ironically, according to Brightmail technicians, their monitoring of e-mail shows that spammers are complying with these new standards much faster than the population as a whole -- more than 12 percent of spammers versus less than 2 percent of all e-mail users.

Even if everyone agrees on a single Sender ID framework, there's no guarantee it will stop spam. Instead, it just verifies the identity of the e-mail sender. Assuming that spammers don't find a way to circumvent the standard before it's agreed upon, it will be easier to track down spammers and prosecute them if they violate the CAN-SPAM act.

Other organizations are also trying to help stop spam. For instance, a number of volunteer organizations maintain lists of spammers or suspected spammers, called black hole lists, or RBLs (real-time black hole lists). These lists are made up of e-mail servers that send spam, e-mail servers that will relay e-mail from anyone, or e-mail servers that are running on cable modem networks.

The intent is laudable, but the results are sometimes less than perfect. These organizations are not responsible to anyone. They are volunteer outfits offering free services, and they can decide on a whim whether to blacklist companies or individuals who may not have anything to do with spam. Many of the e-mail servers on these lists may not be actually sending spam. They may be servers that only have the potential to be used to send spam, or servers run by small companies that are trying to save money. Some RBLs, including, have even decided that all users of DDNS are spammers, resulting in all e-mail, spam or not, sent from DDNS domains to users of the RBL being bounced.

Fortunately, there are alternatives to these non-standard standards. Given the performance of the anti-spam gateways in my most recent tests, the ability to filter out 95 percent to 98 percent of spam with few or no false positives is not dependent on either Sender ID or RBLs. -- InfoWorld (US)

Join the CIO New Zealand group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.

Join the newsletter!


Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.
Show Comments