Post-Mortem Culture for Agent Failures: How GaaS Teams Learn From What Goes Wrong

Table of Contents

• Why Agent Failures Need Their Own Post-Mortem Practice
• What Makes an Agent Post-Mortem Different
• The Anatomy of a Good Agent Post-Mortem
• Reconstruct the Run, Not Just the Outcome
• Find the Decision That Went Wrong
• Classify the Failure Mode
• Blameless, But With Teeth
• Closing the Loop: From Post-Mortem to Eval
• Severity, Cadence, and Who Sits in the Room
• Insights Most People Overlook
• References

Why Agent Failures Need Their Own Post-Mortem Practice

Every mature software organization runs post-mortems. The blameless incident review, popularized by Google's SRE practice, is now standard furniture: an outage happens, a team gathers, they write up the timeline, identify contributing factors, and ship action items. It works because traditional software fails in legible ways. A null pointer, a bad deploy, a saturated database connection pool, these have causes you can point at and fixes you can verify.

Agentic systems break that comfortable model. A per-task agent that books travel, reconciles invoices, or triages support tickets doesn't fail like a web server. It fails by being confidently wrong. It fails by doing something plausible that happened to be the wrong thing. It fails intermittently, succeeding nine times and botching the tenth on the same input, because the model underneath is non-deterministic. When you sell that agent as a service, charging per task or per outcome, every one of those failures is a customer-visible defect that erodes the one thing your business runs on: trust.

That's why a GaaS company can't just bolt agent incidents onto its existing ops review. The failures are a different species, and the post-mortem has to be built for them. This is the cultural infrastructure that sits underneath the whole reliability stack, and it's tightly coupled to the rest of this cluster, your eval suite, your observability tooling, your incident severity standard. A post-mortem without those is just a meeting. With them, it's an engine.

What Makes an Agent Post-Mortem Different

Three properties of agentic systems force the post-mortem to evolve.

First, non-determinism. In a normal incident, "reproduce it" is step one. With agents, the same prompt can yield a different trajectory on the next run, different tool calls, different reasoning, a different answer. You can't always reproduce the failure on demand, which means the post-mortem has to work from captured evidence rather than live re-runs. If you didn't trace the run, you may have lost it forever. The reproducibility problem isn't a nuisance here; it's the central constraint shaping how you investigate.

Second, opaque intermediate state. A traditional stack trace tells you exactly where execution died. An agent's "stack trace" is a chain of natural-language reasoning, tool calls, and intermediate observations, none of which raise an exception when they go subtly wrong. The agent that retrieved the wrong document, reasoned correctly over it, and produced a fluent wrong answer never errored. Nothing crashed. The failure lives in the judgment, not the code path.

Third, the silent failure. The most dangerous agent failures produce no error code and often no obvious sign of trouble. The agent confidently does nothing useful, or quietly skips a step, or hallucinates a field. By the time anyone notices, the customer has acted on the output. Anthropic's own guidance on building effective agents emphasizes inspecting the full trajectory rather than trusting the final answer, advice that maps directly onto how you have to investigate after the fact.

Put together, these mean an agent post-mortem is less like debugging and more like an accident investigation: you're reconstructing a sequence of decisions from forensic evidence, where the "cause" is rarely a single line and the system was technically working the whole time.

The Anatomy of a Good Agent Post-Mortem

A useful agent post-mortem has a shape. Skip parts of it and you get hand-waving; follow it and you get a fix that holds.

Reconstruct the Run, Not Just the Outcome

Start with the trace. You need the complete, ordered record of what the agent did: the inputs, the system and user prompts as actually rendered, every tool call with its arguments and response, every intermediate model output, and the final action. If your observability layer can't give you this, your post-mortem is dead on arrival, and that gap is itself the most important finding.

The reconstruction matters because the failure almost never lives where it surfaced. A wrong invoice total might trace back four steps to a tool that returned currency in cents while the agent assumed dollars. Without the full trajectory, you'd "fix" the math prompt and ship a bug that simply moves. Treat the trace the way an aviation investigator treats the flight data recorder: the visible crash is the last event in a long chain, and the chain is where the lesson is.

Find the Decision That Went Wrong

Once you can see the run, locate the specific decision point where things diverged from a good path. Agentic failures cluster around a handful of these: a retrieval step that pulled the wrong context, a tool call with malformed arguments, a planning step that chose a bad sequence, a reasoning step that misread correct data, or a missing escalation when the agent should have handed off to a human.

Naming the exact junction is what separates a real root cause from a vibe. "The agent hallucinated" is not a root cause, it's a symptom. "The agent had no tool to verify the customer's account status, so it guessed, and guessed wrong" is a root cause, because it points at a fixable design gap.

Classify the Failure Mode

Tag every post-mortem with a failure-mode category, and keep the taxonomy stable across the team. A practical starting set:

• Tool-call failure, wrong tool, wrong arguments, or mishandled tool output.
• Retrieval/grounding failure, the agent worked from the wrong or missing context.
• Planning failure, the sequence of steps was flawed even if each step ran.
• Reasoning/hallucination failure, correct inputs, wrong conclusion.
• "Did the right thing wrong", correct intent, botched execution.
• "Did the wrong thing correctly", flawless execution of a misunderstood goal.
• Missing escalation, the agent should have stopped and asked a human.

This isn't bureaucratic box-ticking. Categories are how you turn a pile of individual incidents into a trend. When you can say "40% of last quarter's Sev-2s were retrieval failures," you know exactly where to invest, better grounding, not a smarter base model. McKinsey's research on scaling and de-risking generative AI repeatedly lands on the same point: the organizations capturing real value are the ones that operationalize learning, not the ones chasing raw capability.

Blameless, But With Teeth

The "blameless" half of blameless post-mortems carries over from SRE intact, and it matters even more here. Agent failures are usually emergent, the result of an interaction between a prompt, a model, a tool, and a real-world input that nobody could have fully predicted. Punishing the engineer who shipped the prompt teaches everyone to hide failures, which is fatal when silent failures are already your biggest threat. You want people racing to surface a bad agent run, not burying it.

But blameless does not mean consequence-free. There's a failure mode in post-mortem culture itself: the review that generates a tidy document, a few "we should consider..." action items, and no actual change. The agent version of "with teeth" is concrete: every post-mortem that identifies a real defect must produce at least one artifact that prevents recurrence in code. Usually that's a new eval case. Sometimes it's a guardrail, an added tool, a verification layer, or a forced escalation rule. A post-mortem that ends in prose and good intentions did not happen.

The cultural test is simple. Six weeks after a post-mortem, can you point to the regression test that would catch that exact failure today? If yes, the culture is working. If the only output was a Slack thread and a vague resolve to "be more careful with prompts," you have post-mortem theater.

Closing the Loop: From Post-Mortem to Eval

This is the part most teams under-build, and it's where post-mortem culture stops being a meeting and becomes a competitive advantage.

Every failure that survives a post-mortem should graduate into your evaluation suite. The bad input becomes a test case. The expected behavior becomes the assertion. From then on, that specific failure is guarded automatically, on every prompt change, every model upgrade, every deploy. This is the bridge between incident review and eval-driven development, and it's what makes the practice compound. Your eval suite becomes a living memorial of every way your agents have ever failed, and a wall against repeating any of them.

The model-upgrade case makes the value obvious. When the provider ships a new model version, your agent's behavior can shift in ways no changelog warns you about. If your post-mortems have been feeding a golden dataset of real past failures, a regression run tells you in minutes whether the new model reintroduces an old bug. Without that, you're shipping on hope. This loop is also why your "reliability number", the headline metric serious GaaS vendors increasingly put on their homepage, can actually move in the right direction over time instead of drifting.

There's a sequencing subtlety worth naming. Because agents are non-deterministic, a single passing run doesn't prove a fix. Promote a failure into the eval suite as a case you run N times, and define success as a pass rate, not a single green check. A failure you saw once and "fixed" with a prompt tweak that passes once is not closed, it's untested. Treat the eval pass rate, run repeatedly, as the real exit criterion for a post-mortem.

Severity, Cadence, and Who Sits in the Room

Not every bad agent run deserves a full post-mortem. You need a severity standard tuned to agentic stakes, and for GaaS specifically that means weighting by outcome, not by uptime. An agent that's slightly slow is minor. An agent that confidently sent a wrong financial figure to a customer is a Sev-1 even if every system was technically "up." The cost asymmetry between a false positive and a false negative varies wildly by vertical, a missed escalation in a legal or healthcare agent is categorically worse than the same miss in a content-drafting agent, so your severity rubric should encode that, not treat all wrong answers as equal.

On cadence: run a deep post-mortem on every Sev-1 and Sev-2 within a day or two, while the trace is fresh and the customer impact is current. Roll the long tail of minor failures into a weekly review where you look for patterns rather than dwelling on each instance, that's where category trends surface and where you decide what to invest in next.

On the room: an effective agent post-mortem needs the engineer who owns the agent, someone from the eval or reliability function (an increasingly distinct role inside GaaS companies), and, for customer-facing severity, someone who can speak to real-world impact. Keep it small enough to be honest and large enough to see the whole picture. The goal is never to assign blame; it's to make sure the same failure can't quietly happen to the next customer.

Insights Most People Overlook

A post-mortem with no trace is a confession, not an investigation. Teams obsess over the format of the review and under-invest in the observability that makes it possible. If you can't reconstruct the exact run, every tool call, every intermediate output, your post-mortem is fiction assembled from the final bad answer. The single highest-leverage investment in post-mortem culture isn't a better template; it's trace capture good enough that you never say "we're not sure what it did."

Your best post-mortems come from the failures customers never reported. The incidents that get reviewed are the loud ones, the customer complained. But the silent failures, where the agent confidently did nothing useful and the customer never noticed or never bothered, are both more common and more corrosive to trust. Mature teams mine shadow-mode runs, human-review samples, and confidence-score outliers to manufacture post-mortems for failures that never generated a ticket. If your post-mortem queue is fed only by complaints, you're studying a biased sample of your worst behavior.

The taxonomy is the product. Individually, agent failures look like bad luck. In aggregate, sorted by failure mode, they're a roadmap. The discipline of tagging every post-mortem with a stable category is what converts a scary, unpredictable failure surface into a prioritized backlog. Teams that skip classification keep "fixing" agents one prompt at a time and never notice that 40% of their incidents share one missing tool.

"Fixed" is a pass rate, not an event. Because the substrate is non-deterministic, the deploy-time instinct of "I changed it, it works now" is actively dangerous. A fix that passes once may fail one run in five. Until a failure is encoded as an eval that runs many times and clears a threshold, it isn't closed, and pretending otherwise is how old bugs walk back into production after a model update.

The reliability moat lives in the post-mortem archive, not the model. Anyone can buy access to the same frontier models you use. What they can't buy is your accumulated library of real failures-turned-eval-cases, the institutional memory of every specific way your agents have broken in your domain. That archive is slow to build and nearly impossible to copy, which is exactly why it, and not raw capability, is what separates the GaaS companies customers keep paying from the ones they quietly churn away from.

References

• Building Effective Agents, Anthropic
• The State of AI, McKinsey & Company
• Google SRE: Postmortem Culture, Learning from Failure