Most of us know that there is no such thing as 100% security, and that – unfortunately – it’s only a matter of time until a security incident occurs.
Despite this, it’s rare to see a good incident response process and plan in place.
When an incident occurs, it’s more common to observe frantic running around and (admittedly smart) people improvising action plans.
The lack of upfront preparation can lead to disaster, as bad decisions get made under pressure.
Incident handling is a vast topic, but here are a few tips for you to consider in your incident response. I hope you never have to use them, but the odds are at some point you will and I hope being ready saves you pain (or your job!).
- Have an incident response plan. It is, of course, the most obvious advice, however you would be amazed at the number of organisations that “haven’t quite got around to it yet”.
It doesn’t need to be 400 pages long (nor should it be half a page most likely) but documenting your incident response plan and distributing it up front can make a massive difference when the inevitable occurs.
- Pre-define your incident response team and make sure it draws from multiple disciplines. A good incident response team should not just consist of IT or security people (though they undoubtedly need to be a strong core), but should also include PR, HR, Legal and executives to name just a few.
For instance, not involving the PR team could result in external communications that are more damaging than the breach in the first place.
Make sure each member of the team is briefed on their involvement and expectations. Watch that list of team members, however. It gets old quickly as people leave, go on vacation or just forget what they signed up for.
- Define your approach: watch and learn or contain and recover. We have seen great examples of this in the news recently. When an incident occurs and is verified – for instance a hacker compromising one of your web servers – you need to make the call on whether to recover as quickly as possible or to watch the attacker and learn.
You need to make this policy decision upfront because it requires good preparation, executive support and a skilled team to manage the risk. Most organisations will (and should) focus on containing the damage and recovering business systems.
Detailed forensics and observing the attackers is often too great a business expense for many firms, but don’t give up entirely on capturing evidence – you never know when you may need it later.
If you are a target likely to come under persistent long term attacks from adversaries (perhaps a nation state, a competitor or a hacking gang that you have riled somehow) learning more about your adversary to help build future defences can make a lot of sense.
You should not venture down this path unless you have the resources (people, duplicate systems, network infrastructure for suitable containment) to execute it successfully and it absolutely requires executive buy-in or you will find yourself out of a job very quickly if it goes wrong.
Make this decision up front, discuss the risk attitude of the business and alter your incident response process appropriately. Of course, you won’t watch and learn on every attack, so identify the two paths and when you will use them.
- Pre-distribute call cards. Another common mistake is to depend upon your normal communication infrastructure in the event of an incident.
Imagine that the LAN stops working, no-one knows anyone else’s number and email doesn’t work. It will be pretty tough to handle an incident in that situation.
Decide up front communication methods, choose a call bridge or similar and then distribute details to each of the stakeholders.
- Forensic and incident response data capture. This goes wrong a lot. During and after security incidents, pressure can be high and there is a tendency to rush, which – of course – means mistakes are made.
You need to ensure that you have comprehensively captured the right data and logs without taking up hours you don’t have – it’s not a trivial balance.
In particular, define how you will capture notes (I like a lined, dated paper notebook which is easy for courts to get their heads around) and evidence.
Do you run Volatility to capture memory? Do you power off the machine and take an image of the disk to capture as much as you can before the attack shuts down? Decide in which instances you will follow each path and build a toolkit ready to go.
- Get your users on-side. Incident response is not just an ‘IT thing’. Make sure your incident response links up with your organisation’s acceptable use policy and security awareness program.
You want your users to know how to tell you if they think an incident is occurring. In particular, system administrators, application owners and data owners should know how to contact you if they spot something unusual. The incident response process can then be spun up (or spun down if it’s a false alarm) quickly.
Don’t just write the policy and leave it on the intranet somewhere.
- Know how to report crimes and engage law enforcement. Certain types of incident may require you to report to law enforcement but often when such an opportunity arises no-one knows exactly how to do it.
From having your web server hacked to the theft of credit card details or IP it pays to understand the process and requirements in advance. Check out Naked Security’s quick guides on reporting computer crimes and find out who your representative is before you need them.
- Practice makes perfect. The process can be documented but when it comes time to use it the bridge is enabled and no one connects or knows what to do. Stage an incident and have your team dial in and work through the mock scenario.
Of course, be sure to make them aware that it is a drill or you might end up in the awkward scenario of announcing a practice-run to the real world as if it actually happened. It’s hard to convince people you weren’t hacked retrospectively!
Happy incident handling.