I have been attending the Virus Bulletin conference in Geneva Switzerland for the first time this year. This morning I watched Andreas Marx and Maik Morgenstern of av-test.org deliver a talk entitled “Why ‘In-The-Cloud’ Scanning is not a Solution”
They presented their results of some in-depth testing of anti-virus solutions using the cloud as a supplemental method to deliver malware identities. What did they find?
Overall they determined that solutions using “in-the-cloud” services were no more effective than traditional anti-virus solutions. They also noted that the results they have seen from the vendors they tested are wildly unpredictable as to what to expect from one threat to the next.
One of the points made by Andreas really hit home for me, which is that the ability to publish identities seems to be the gating factor in providing up to the moment protection. The delivery mechanism is largely unimportant if you have a reliable means of providing threat data to the product.
In their paper, they also mention that on-computer anti-virus has far more capabilities for detecting new malware than simple file checksums. Today’s cloud-based services rely on checksums which are not equipped to deal with server-side polymorphic malware.
Andreas had pointed out that there were inconsistencies with results from some vendors. He showed how one vendor showed a file as being suspicious via its cloud service, then it was safe later that day, and finally marked malicious that evening. During the question and answer period, Dmitry Gryaznov had some clarifications to this slide from McAfee’s perspective. Confusingly, Dmitry seemed to confirm that this was in fact true.
Another issue raised in the talk was around network impact, especially in organizations with a large concentration of computers. Not just malicious files are being checksummed and sent into the cloud, many legitimate files may trigger the technology as well. In their paper, they point out that these transactions can be 5K bytes or more, resulting in a potentially significant amount of bandwidth in a organization with network capacity issues.
Unless I misunderstood, this rush to identify checksums and publish them as suspicious and revoke them later seems to imply that there could be a high false-positive, or false-negative problem. Andreas and Maik touched on their concerns related to quality assurance processes as well.
The conclusion of the tests performed reinforced my existing thoughts on providing best protection to our customers computers. Provide quality updates as fast as you can. The means of delivery are not important so long as the computers receive their identities.
Sophos has used “the cloud” in our anti-spam solutions for several years, and like any other technology will carefully consider which tool provides the best protection for our customers in each scenario we provide solutions to.
Creative Commons cloud image courtesy of EssjayNZ’s flickr photostream
Update: Paul Ducklin of Sophos Australia has published a podcast covering cloud computing in more detail.
Gurusamy Sarathy also provided me another outlook on cloud security. He pointed out that anti-virus vendors struggling with increased update sizes can use the cloud as a way to reduce the amount of redundant identities being published to individual computers.