The First Five Minutes of an Incident
This article explores what the first five minutes of a major incident reveal about how well a team actually recovers.
9 minute read
2:13 AM.
Phones start ringing before most people are fully awake. Alerts are firing across three different systems at once. A team channel that was silent an hour ago now has forty unread messages, and half of them contradict each other. Someone is already typing “is anyone else seeing this?” into a channel where six other people are typing the exact same question.
Nobody knows yet if this is a server, a network problem, a vendor outage, or something worse. What everyone in that channel does know, instantly and without being told, is that whatever they say in the next few minutes is going to matter. That knowledge alone is enough to make some people freeze and others start moving too fast in the wrong direction.
Most conversations about incident response skip straight to the technical fix. They talk about monitoring dashboards, runbooks, escalation paths. All real, all useful, and all beside the point for what actually determined the outcome in every major incident worth learning from. That got decided earlier than the fix, in the five minutes before anyone had a real answer.
So: what should actually happen during those five minutes? Not what the runbook says should happen. What actually separates a recovery that takes twenty minutes from one that takes four hours, before a single root cause has been identified.
The Biggest Mistake People Make
The instinct, almost universally, is to start fixing the technology immediately. Someone restarts a service. Someone else rolls back a deployment that may or may not be related. A third person starts pulling logs, convinced they’ve already found the cause.
None of that is wrong, exactly. It’s just premature. Fixing something you don’t yet understand isn’t troubleshooting. It’s guessing with more steps, and guessing under pressure has a way of turning one incident into two.
Picture a version of this that plays out constantly. An alert fires for a service that’s returning errors. The instinct is to restart it. A reasonable first move, most of the time. But if that service was returning errors because a database it depends on is overloaded, restarting it doesn’t fix anything. It just adds a second wave of connection attempts on top of an already-struggling database, at the exact moment it needed less load, not more. Ten minutes later, two systems are down instead of one, and three different people are each convinced they’ve found the real cause, because each of them changed something at roughly the same time.
The teams that recover fastest aren’t the ones who start typing commands first. They’re the ones who spend the first sixty seconds doing something that feels, in the moment, like it’s costing them time: figuring out what they’re actually looking at before they touch anything.
The first five minutes aren’t for fixing. They’re for finding out what you’re fixing.
“The first five minutes aren’t for fixing. They’re for finding out what you’re fixing.”
Stabilize Before You Solve
Before anyone reaches for a solution, four things need to happen. Not in a rigid sequence, but all of them, early:
- Confirm impact: what’s actually broken, and for whom, right now
- Establish ownership: who is running this, so two people aren’t quietly making different decisions
- Open communication: one channel everyone trusts, instead of five conversations that don’t know about each other
- Avoid assumptions: the difference between “we think” and “we know” matters more in minute three than it will in hour three
Confirming impact sounds obvious until you watch a room skip it. “Everything’s down” and “the checkout page is slow for some users” call for completely different responses, different urgency, and different people in the room. Without confirming which one is actually true, teams routinely spend twenty minutes preparing a response sized for the wrong incident.
Ownership is the one people resist most, because naming an owner in the first sixty seconds feels premature when nobody yet knows what’s wrong. It isn’t. The incident commander’s job in minute one isn’t to know the answer. That’s the person everyone else defers to about what happens next, so the room doesn’t end up with three well-meaning people giving three different instructions at once.
None of these four things fix anything. That’s the point. An incident with clear ownership and an honest picture of impact, still fully broken, is in a fundamentally better position than one that’s half-patched by three people who don’t know what the other two just changed.
Stabilizing isn’t slower than solving. It’s what makes solving possible instead of accidental.
Questions Worth Asking Before Any Answer
A short list, worth having ready before you need it rather than improvising it at 2 AM:
- What changed? Most incidents trace back to something that changed recently, not something that simply failed
- Who is affected? A handful of internal users and your entire customer base call for very different responses
- Is this isolated, or is it still spreading? The answer changes whether you’re debugging or containing
- Do we know, or are we guessing? The single most useful sentence anyone can say out loud in an incident channel
“What changed” deserves to be asked before almost anything else. Systems that have been stable for months rarely fail spontaneously at 2 AM on their own. A deployment, a configuration change, a certificate expiring, a vendor’s own incident. Something usually moved. Finding that something is often faster, and more reliable, than trying to reason your way to a root cause from symptoms alone.
“Is this isolated, or still spreading” is the question that decides whether you’re solving a problem or racing one. A contained issue and an actively spreading one require different postures entirely. One rewards careful diagnosis; the other rewards fast containment even before the cause is fully understood. Confusing the two, treating a spreading incident with the patience appropriate to a contained one, is how a bad hour becomes a bad night.
That last question, do we know or are we guessing, does more work than any of the others. Saying “we’re not sure yet” out loud, in front of the group, is uncomfortable. It’s also usually the moment an incident stops getting worse, because it’s the moment the guessing becomes visible enough for someone else to correct.
Here’s a version of that worth recognizing: someone says, with real conviction, “this started when the deployment went out at 1:47.” It sounds like a fact. It might be an observation. The timing lines up, nothing more has been confirmed yet. Said as a fact, it quietly redirects the entire room toward that deployment, and everyone stops checking other possibilities. Said as “the timing lines up with the 1:47 deployment, but we haven’t confirmed that’s the cause,” it does the opposite: it points the room in a direction while keeping the door open to being wrong. The difference in wording costs nothing. The difference in outcome, if that lead turns out to be a coincidence twenty minutes later, can cost quite a lot.
The First Five Minutes
The Psychology of Those Five Minutes
Technical skill matters. It’s not what separates a fast recovery from a slow one, not in the opening minutes.
A brilliant engineer who panics will, almost every time, be outperformed by a calmer one who is merely competent. Panic doesn’t just feel bad. It actively degrades the thing an incident depends on most: clear thinking under uncertainty. People talk faster and listen less. Assumptions get stated as facts because saying “I don’t know” feels, in the moment, like a weakness instead of the most useful thing anyone could say.
There’s a specific version of this worth watching for: the person who is technically right, loudly, at the wrong moment. Being correct about a detail three steps ahead of where the group currently is doesn’t help. It fragments the room’s attention and makes the actual sequence of decisions harder to track. Good incident leadership isn’t about being the most technically capable person in the channel. It’s about keeping the group’s attention on one question at a time, in the right order.
Calm is not the absence of urgency. It’s the discipline to feel the urgency and still ask the right question before reaching for an answer. That’s a skill, not a personality trait. It can be built, deliberately, before the incident that requires it. People who run tabletop exercises, walk through past incidents afterward, or simply practice narrating their own thinking out loud tend to arrive at that calm faster than people who are encountering the pressure for the first time live.
Calm leaders outperform brilliant technicians who panic. Not because they know more, but because panic makes everyone, including experts, temporarily worse at knowing what they know.
Where AI Actually Helps Here
Not by deciding. By gathering evidence faster than a person can.
Pulling logs across systems, correlating timestamps, summarizing what changed in the last hour, surfacing the three most likely related deployments. That’s exactly the kind of work AI tools are good at, and exactly the kind of work that used to eat the first several minutes of an incident by hand. A summary that would have taken a person ten minutes to assemble by grepping through five different systems can now surface in under a minute, and that minute matters.
What it shouldn’t do is make the call. “Is this isolated?” and “who is affected?” are judgment questions, informed by evidence but not answered by it alone. An incident commander who lets a tool’s confident-sounding summary substitute for their own judgment has just reintroduced the exact problem Article #2 spent three questions warning about. A model can tell you what changed. It can’t tell you, with the accountability a real incident requires, whether that change is actually the cause. A wrong answer delivered with total confidence is worse than no answer at all, because it stops the room from asking the follow-up question that would have caught it.
The practical version of this: treat AI-generated summaries the way you’d treat a report from a junior team member you trust but haven’t fully verified yet. Useful, often accurate, worth acting on quickly, and still something a human confirms before it becomes the official account of what happened.
Used well, AI compresses the evidence-gathering that used to take five minutes into thirty seconds. That doesn’t eliminate the need for stabilizing before solving. It just means there’s more of those five minutes left over for the judgment that actually needs a person.
Understand First. Decide Well.
Every incident eventually gets a technical explanation. A server failed, a deployment broke something, a vendor had an outage. That explanation is almost never the whole story of why recovery took as long as it did.
The 2:13 AM version of this always feels unique in the moment: the specific system, the specific alert, the specific person who happened to be on call. It rarely is. The pattern underneath it repeats. Someone reaches for a fix before understanding the problem. Ownership stays ambiguous a few minutes too long. “We think” quietly gets treated as “we know.” None of that shows up in the postmortem’s technical summary. All of it shows up in how long recovery actually took.
The best organizations don’t recover faster because their technology is better. They recover faster because they make better decisions before they know the answer.
Understand first. Decide well.
Before You Decide
Five questions worth sitting with before the next incident, not during it:
- Do I know who owns an incident the moment it starts, or does that get figured out live?
- Would I recognize the moment I’m guessing instead of knowing?
- Am I comfortable saying “we’re not sure yet” out loud, in front of a group, under pressure?
- What am I already using AI for in an incident, and is it gathering evidence or quietly making decisions?
- If I’m honest, do I default to fixing first or understanding first?
Continue Your Thinking
One practical lesson, every week.
Written for people responsible for technology decisions. No hype, no roundups, no sales sequence.