In a previous post we discussed Incident Response (IR) processes and our preference is one that runs Preparation, Identification, Containment, Eradication, Recovery and Lessons Learned (PICERL). In this post, we are going to be looking a bit more into the final phase of IR – Lessons Learned.
As the final step, this is probably the IR phase that gets most overlooked. But it is one of the two most important. After the excitement and stress of clearing out an incident, it is really important that you learn. Overlooking this step inevitably leads to repeating problems. First you never really solve the problem that let the attackers in. Secondly, any issues your responders faced remain. Together this means you see more incidents and they take longer to resolve.
Learning from experience really is important.
Lessons Learned
This phase goes by many names. For example, you might call it a “debrief” or a “wash up”. The exact name isn’t as important as making sure you do it. However, it is also important that this is a learning phase. Avoid falling into the trap of blaming people. Also, avoid any hint it is a witchhunt or an attempt to sack someone. This can be hard, as sometimes a person is responsible. But at a practical level, this should be something outside the IR process.
In this phase, you are trying to answer the following questions.
- What happened?
- How did it happen?
- How did we deal with it?
- What went well?
- What went badly?
- Most importantly, what do we need to change?
Now it is totally down to your organisation how formal you want this to be. What matters above all else, is that you go through this. For example, you might decide that major incidents need a formal report and review. But then decide that minor events simply need an email update. But other organisations might want a formal report for every incident. Remember, though, that this can be resource-intensive so keep in mind how many incidents you have when deciding this.
The ultimate goal of this phase is to understand what happened. That is really all that matters so the specifics can be quite “personal.” It certainly helps if, during planning, you decide how this phase will be run. In the same vein, planning allows you to make sure you capture the right data. Without the right data, you can’t learn, so you can see the problem here.
Take Note
One big caveat to remember is that sometimes the lesson is trivial. For instance, with a single malware event, you probably dont need a full report.
Also, try to focus on how to improve rather than punish. If your lesson learned is that a person messed up, this isn’t helpful. Instead, you can focus on the need for more training, or better still, improved processes. You might feel people are to blame, but most of the time that is only because technology or process failed.
Summary – what have we learned?
In short, the lessons learned phase is critical to good IR practices. However, it entirely depends on your planning and IR data collection. In addition, the exact process should depend on your specific needs and the nature of the incident. Some organisations have a formal process with pre-designed data capture forms. Others go for an ad-hoc meeting. The important thing is making sure it happens and doesn’t simply burn resources.
If you want to know more, or get some help with an incident or even building your own incident response process then get in touch and our consultants will be happy to assist.
Great series of posts – super useful to see it explained in steps. I’ve found this phase, “lessons learned” is nearly always skipped. It causes chaos.
Yass – we call it “hot debrief” and it happens on about 1 in a hundred incidents. Then people get uptight about the same things causing incidents over and over…
“Those who cannot remember the past are condemned to repeat it.”
George Santayana