Common problems plague our operations and exercises
By Kenneth Van Wyk (KRvW Associates, LLC, US) and Elliott Atkins (Exercise3, UK)
July 16, 2026
Greetings FIRST Friends
Thanks to all of you who joined us in Denver and attended our session where we presented our eight most common incident response problems we have individually observed over the years, both in exercises and operations.
We promised we are opening a door to a discussion, and we’d like to invite our FIRST colleagues to join in.
You see, our findings come from direct observation of hundreds of operations and exercises. While I’d stop short of calling it a truly scientific study, our findings are pretty darn objective – and they were startlingly similar between our two data sets, and those two data sets are pretty much 100% isolated from one another.
In Denver, we briefly covered our recommendations for remediating those eight most common problems we observed. Unlike our objective data, our remediations are our opinions. We believe those opinions are helpful, but that’s where we would love to hear from our colleagues and friends in FIRST.
If you’ve seen the same sorts of problems in IR, how did you or your clients remediate them? Which remediations worked best? Which did not? For that matter, if you observed major problems that aren’t on our list, we would like to hear from you.
Please do take a critical view to our eight findings. Once you’ve considered those, please join us in discussing which remediations work best, and for whom. We expect there will be many solutions, and some solutions will work better for some organizations and less so for others. But, we want to hear from you.
If you care to share your opinions – no client names will be shared – please contact us and let’s talk.
Surprises
There were some surprises in our work. Effective incident response requires technical expertise on many issues, from digital forensics to malware analysis through reverse engineering of encrypted machine code. We’ve found that almost all teams possess the necessary skills, either in-house or via vendor support, to get the technical job done effectively.
Similarly, effective incident response usually requires deep subject matter expertise on the parent company’s business applications, whether they are developed in-house or software as a service (SaaS) applications. Here too, almost all teams are able to muster the business and application knowledge needed to solve difficult problems regarding those business applications.
With this complement of technical skills and business knowledge, many teams are able to do good things when it comes to incident response operations (IRO). We’ve seen many teams that consider that to be adequate.
On the other hand, many of the mistakes we’ve seen and have highlighted below, can be costly and can harm a company’s business continuity as well as its reputation. Let’s dive in.
Our Findings and Recommendations
These are the highlights of what we’ve directly observed, along with brief recommendations. We’ll be expanding on each of the recommendations in the coming weeks.
Incident Response Plans
Every team has an incident response plan (IRP). Some of them are short, but all too often, they are monster documents that have grown over time to cover every possible security topic. The expectation is that everyone concerned will have read and truly internalized the IRP, and will thus know exactly what he is to do during an incident. This is simply not the case in practice.
Clearly, not everyone needs to read the entire IRP. In fact, most of the people necessary to effective IRO should not have to read the IRP at all. They do, however, need clear guidance on what they need to do during a crisis in a quick, easy-to-digest format. Process diagrams and checklists tend to be far more effective than an enormous unwieldy plan.
Process Errors
Despite many teams being able achieve some good things via an ad-hoc, intuition-driven process, those approaches also often result in mistakes being made, some of which can be costly or impossible to correct after the fact. Common examples of this include failing to ensure incident-relevant information is protected as potential evidence in a criminal or civil legal matter, failing to effectively track costs, and failing to comply with the cyber security insurance provider’s requirements.
In all likelihood, these things were well documented in the IRP, but neglected during the heat of a stressful incident operation. Someone needs to manage these details to ensure they’re being done.
Failure to Prioritize Business Impact
Most IRPs include an incident severity determination, usually coming from a triage process of some sort. All too often, though, the severity tiers do not adequately capture the real or potential impact to the business. Also, the severity level is not a simple static metric. It can evolve and become worse or — less likely — better over time.
Ensuring the business needs are properly included and addressed in the triage process is vital.
Uncertainty About Decision Authority
Figuring out during an IRO who should make those difficult decisions is not the time to do it. Incidents don’t wait for us, and we need to have the ability to gather and assess the facts as we know them, and then make quick decisions when the situation calls for it. That means we need to understand just who makes various types of decisions, and those people need to be deeply and actively involved in the process.
Unavailable Key Personnel
It’s easy to dismiss this as being exclusively related to tabletop exercises not being taken seriously, but we’ve also seen the same problem during many live incidents. The fact is that many key roles are usually spelled out in the IRP, and those people need to be accessible. When they are not available, they need to have secondary staff who are available and authorized to act on their behalf. Again, incidents are not patient.
Failure to Reach Decisions
Even when teams know who is in charge of a particular issue, we’ve seen time and again situations where challenging topics are discussed in detail, but then no decision was made. Everyone chimes in with his or her opinion, and everyone gets heard. But, we’ve seen many situations where those topics are challenging ones, and the group quietly “moves along” without having come to a decision.
Sometimes, difficult decisions simply have to be made, and in a timely manner. Irrespective of who makes them, failing to do so at all is often just not enough.
Action Assignment and Tracking
When incident response teams (IRTs) operate in an “everyone go deep” manner, it is really easy to lose track of who is doing what. Similarly, assigned actions need to be clearly understood by all parties.
Someone needs to actively manage the process and be able to assign and track activity across the various strands of incident response activity.
Ineffective Communications
All of the things we’ve already discussed can be undermined — and frequently are — by failing to communicate clearly. An IRO is no time for imprecision or ambiguity in our communications. It’s not just a matter of who is doing what, but does the person you just asked to perform a task clearly understand what is to be done? How do you know?
We've also seen difficulties occur when teams need to explain the incident to a non-technical audience. Briefings packed with technical jargon, or "down in the weeds" detail, can lead to confusion and alienating key decision makers.
Final Thoughts
This collection of common flaws is not exhaustive or conclusive, but it does accurately reflect what both of us have directly observed in our two sample populations, and over a significant period of time. It is no coincidence that our findings so closely mirrored each other, despite our groups being all but completely isolated from one another.
We’ve alluded here to some remediations, but the biggest and most effective fix is for someone to be deeply active and immersed in the IRO process, and to orchestrate it like a symphony conductor. That person should be on top of every aspect of an operation. That person should be a highly effective communicator and be able to operate reliably and repeatably under enormous stress situations.
Catch the talk at FIRSTCON26
