Is my spam count bigger than yours?

Are you getting enough spam?

Measuring how good is your spam protection is something that can be very hard to quantify. By its nature spam is very changeable. This morning’s spam has been replaced with something new by lunchtime. Does it make sense testing an anti-spam product using a spam email from last week. Here at SophosLabs we receive in excess of 25 million spam emails a day on our spam traps. What testing organisation is going to be able to retain and use this volume in testing?

Several testing organisations do conduct anti-spam tests which are a good comparative measure of how good one product is compared with another. My personal method of measuring the quality of protection is to record just how many spam emails a day get through the anti-spam product that we use, which just happens to be our very own PureMessage.

As a longer serving employee of Sophos I have the dubious honour of being one of those people whose email address has been publicly listed on various sites since before spam became the problem that it is today. Newer members of staff hardly receive any spam because their email address is not public and has never been harvested.

As a result of my email address being public I receive a steady stream of spam, the bulk of which is quarantined by PureMessage. Since July 2007 I have been recording just how many spam emails have been trapped and how many have got through to my desktop and the graph below shows the data I have captured.

Spam volume from July 2007 to Jan 2008 was reasonably even with an average of 100 messages a day being blocked. However since Jan 2008 there has been a steady increase and I am now seeing around 150 to 160 emails a day being stopped. There is no obvious pattern to this. Trawling through the quarantine area of PureMessage has not shown an obvious growth in any particular spam but just an overall increase of all categories.

The other point of interest is the change in how many emails get through to my desktop. From Jul 2007 to Nov 2007 the average number of spam emails getting through to my desktop was 3 or 4. Since November it has been down to between 1 and 2 emails per day. There are also now days when there are no spam emails getting through at all. Even the spikes on poor days are not as big as before. This does coincide with the launch of SXL, our latest method of delivering protection.

On reflection, your typical user probably does not really care how much spam is stopped, that is of interest to those of us who want to know how much spam is out there. I believe your typical user is only concerned with how many spam emails a day get through and on current performance I am pleased that I am now down to just 1 or 2 spam emails a day. I just wish they could be more interesting!