Proactive detection of unknown malware – a real test of anti-virus software

Sophos’s Behavioral Genotype protection is one of the technologies built into Sophos’s threat detection engine to protect proactively against new, unknown malware before we have analyzed it in our labs. In these days of commercially-driven cybercriminal gangs and zero day exploits, it’s an important part of the defense we can offer our customers.

The inevitable question I am then asked, however, is “Just how good is Sophos’s proactive protection?”. Around 85% of the new malware samples we receive each week in SophosLabs are proactively detected in one way or another, but what does that actually mean, and how does it compared to our competitors in the computer security industry?

Of course, it’s relatively easy to “prove” that one anti-virus technology is better than another, when it is you that carries out the malware detection test and you which decides the methodology used in the review! Producing more credible and reliable evidence is more of a challenge.

For that reason we turned recently to AVIEWS. AVIEWS is the Anti-Virus Information & Early Warning System, an online community that shares information about breaking malware threats between anti-virus vendors, researchers, security experts, and system administrators. If you’re responsible for computer security inside your company you may want to look into joining AVIEWS and its sister organization, AVIEN.

AVIEWS has a variety of mailing lists to discuss different topics. One of those mailing lists is a forum for members to send out alerts of new malware that has been spotted, warning other members of its existence. When these alerts are sent out, members usually include the current detection status in the form of a report from virustotal.com which details which security vendors are currently detecting a particular outbreak.

I decided to look at the last four months of alerts posted to the AVIEWS mailing list, and compare our proactive detection rate to other vendors. Out of 20 alerts, proactive detection results were as follows :

PROACTIVE DETECTION OF NEW MALWARE

               ABCDEFGHIJKLMNOPQRST  Total  %
AhnLab-V3      .................x.x    2    10%
AntiVir        x.xx.x..x.xxxxx..x.x   12    60%
Authentium     x..x.x......x.x..x.x    7    35%
Avast          x.....x.x..x.......x    5    25%
AVG            x.xx..x.x.xx..xxx...   10    50%
BitDefender    x.xx..x.x..x..x.....    7    35%
CAT-QuickHeal  x.xx..x.x..xx.x.....    8    40%
ClamAV         xx....x....x.xx..x.x    8    40%
DrWeb          xxxx..x.x..xx.x..x..   10    50%
eSafe          x.xx..x.x..xx.x....x    9    45%
eTrust-Vet     .xxx.......x..x.x...    6    30%
Ewido          ....................    0     0%
FileAdvisor    ....................    0     0%
Fortinet       xx.x..x.x...........    5    25%
F-Prot         x..x.x......x.x..x.x    7    35%
F-Secure       ..xx.x.xx..xx.x..x.x   10    50%
Ikarus         xx.xxxx.x..x..x.xx.x   12    60%
Kaspersky      x..x.xx.x.xxx.xxxx.x   13    65%
McAfee         ...x..x.x..x..x....x    6    30%
Microsoft      x.x...x.x.xx..xxx.x.   10    50%
NOD32v2        .x.x..x....xxxx.xx.x   10    50%
Norman         ..xx...x...xx.x..x.x    8    40%
Panda          ............x.x.....    2    10%
Prevx1         .............x......    1     5%
Rising         x..........x..x....x    4    20%
Sophos         x.xxxxx.xx.xxxx.xxxx   16    80%
Sunbelt        x.xx..x.x...........    5    25%
Symantec       -.xx.x-.x.xx..x.....    7    35%
TheHacker      -..x..-.x..x.....x.x    5    25%
VBA32          -...x.-........x....    2    10%
VirusBuster    -.xx..-....x..x..x..    5    25%
Webwasher      -.xx.x-.x.xxxxx..x.x   11    55%

Key
---
x : detected
. : not detected
- : VirusTotal did not produce a result

As you can see Sophos’s Behavioral Genotype protection performed very well, successfully detecting more of the unknown malware than any other product tested.

These findings echo those of Cascadia Labs who recently published a comparative review of Sophos, Symantec and McAfee’s protection against zero-day threats:

We’re not resting on our laurels though – you can expect us to further enhance our ability to proactively protect computers against zero-day threats and unknown malware in 2008 and beyond.

Please note, that this analysis was carried out based on alerts received from AVIEWS and was NOT a test carried out by AVIEWS