In my day job as a tester of anti-malware solutions, for some time now I've been getting asked the same question by all sorts of people. They want to know how I plan to test against Advanced Persistent Threats, aka APTs.
It may seem like an obvious question - APTs are considered a major issue affecting larger organisations, they generally involve malware of some sort, so we should be able to compare how well different products handle them.
But targeted attacks are very different from your everyday malware, and testing protection against them turns out to be a very different kind of task.
Advanced Persistent Terminology
The phrase "Advanced Persistent Threat" has been around for some time.
The most commonly cited origin story dates back to 2006, when US Air Force researchers are said to have coined the term.
The term was immediately and enthusiastically taken up by media and marketing types, and as such was of course roundly dismissed by more technical folk as a mere publicity buzzword, quite unnecessary when the standard term "targeted attack" sufficiently covered the threats involved.
It has since been fairly generally adopted to refer to crafted, long-term attacks on larger organisations.
When Stuxnet was discovered, it became an archetypal APT.
It was aimed at a specific well-protected target, generally agreed to be systems controlling Iranian nuclear processing; it was tailored to penetrate that target, by using USB thumb drives to jump air gaps; and it had a payload designed to have a particular effect on that target, apparently to skew the running of fuel-processing centrifuges and potentially destroy them.
Perhaps it is this association which has confused the issue. The Stuxnet seen by the world was a specific item of malware, and it seems in many minds the idea of an APT has remained that simple - a single executable which can break in somewhere, find its way around and carry out its mission.
The reality is that most APTs, Stuxnet included, are more like long-term team operations than one-off, one-man attacks. They will involve many different people doing different things at different stages, with all their actions and the data they gather interweaving to produce the end result.
Some of the stages will involve actual items of malware, some will require more hands-on, real-time hacking and exploration of backdoored networks, and some will be much more human, such as physically poking around a building that houses systems which may be targeted.
Attacks and Prevention Techniques
The threats "targeting" home users and smaller businesses are not really targeting you at all - they don't care who you are or what you do, they just want to break into your computer and harvest your bank login details, sniff for payment transactions, lock up your files and demand a ransom, or simply rent out time on your machine to spammers.
So, you can choose a general-purpose anti-malware solution, and can base your choice on standard factors like price, support quality, and performance in comparative tests. A product which provides a better range of protection against all sorts of attacks offers you a better chance of avoiding being hit by anything.
Larger business and institutions are in very different positions though.
Everyone from retailers to software houses to defense contractors to stock exchange operators will be running much bigger, more complex and more valuable networks, across multiple sites, with data and capabilities which could be hugely valuable if they fell into the wrong hands.
Of course, they still need to protect against the everyday broad-spectrum malware, and will probably be relying on high-quality anti-malware to help them do so.
They'll probably have multi-layer protection at the desktop and at gateways. They'll have used independent tests to help choose which products to use too, perhaps weighing up both comparative and certification data.
Of course, for them, factors like support and scalable management facilities will have considerably more weight than they would for small firms or individuals.
But they'll also have a wealth of other security measures too. Sophisticated firewall rules, demilitarized zones, intrusion detection systems, data leak prevention, careful assignment of access rights, vetting of staff, contractors and visitors, and all manner of physical security measures all come into play when it comes to blocking targeted attacks.
The point about carefully targeted attacks, including APTs, is that they will take all these factors into account when designing their attack strategy.
If they know you have bulletproof glass, they're not going to try throwing a brick through a window; if they know your AV's behaviour monitor alerts on certain registry changes, they're not going to make those changes until they've disabled or subverted the AV.
Actual Protection Testing
So how can we test how well anti-malware solutions perform against targeted attacks?
Sure, we can take a bunch of files known to have been used in targeted attacks in the past, and compare detection of them by various solutions, but this will have little value.
We will never be able to say "Product X would have protected you against Attack Z", simply because if you had been using Product X rather than Product Y, Attack Z would have taken a different form. It would have been tweaked to ensure it got around any barriers Product X put up.
One thing we could do is to try to figure out how easily products can be bypassed or subverted, by creating new malware or attacking components of the solution.
The morality of the first is dubious (anti-malware firms hate it when you create new samples for them to deal with, given the amount of real-world stuff turning up every second), and the second is, one would hope, performed all the time as part of in-house QA and hardening.
The efficacy of both as a test metric is also suspect. We could put together a list of a dozen ways of taking down a solution, and rate products on how many they proved immune to, but there's no saying that the guys behind Attack Z won't come up with another method we never thought of, use it as their first attempt and successfully beat the product we rated top.
The whole point of these holes is that no-one thinks of them until someone thinks of them.
The only real way of measuring resilience to APTs is penetration testing of a fully-secured site or system, checking every potential hole you can think of to see if it's covered, but again this relies on the pentester being at least as knowledgeable, well-resourced and creative as any possible attacker.
This sort of testing is also not really useful as a guide for others on what to do. Even if you could sum up all the measures in place in a site found to be fairly secure, it would be near-impossible to ensure you could reproduce that exact setup in a different place with different people, needs and behaviours.
Avoiding Potential Tragedies
There are all manner of solutions offering to protect from APT attacks, and for all of them it is similarly difficult to validate their performance thanks to the unknowable nature of potential attacks.
No matter how tough a set of locks you put in place, if you have something someone wants badly enough, they're going to keep on picking away until they find a way through.
Even if your technological protections do prove too tough to crack there are likely to be other vectors - there aren't many products promising to keep your sysadmins safe from blackmail or hypnosis, for example.
In the end, the best way to make sure your defences are as strong as they can be is to follow established best practices, using the most comprehensive set of protections and making sure they are properly implemented and maintained.
Each time a new penetration takes place, at least one which doesn't rely on some sort of error or sloppiness, it's vital that information on what went wrong is shared so that those best practices can be expanded and updated.
When fighting an invisible enemy, it's hard to guard every angle, perhaps even to know all the angles which need guarding, but learning from the mistakes of everyone around us makes us the toughest possible nut to crack.