Why "99% Accurate" Is Meaningless for AI Agents (And What to Measure Instead)

Table of Contents

• The Number That Sounds Great and Means Nothing
• Accuracy Is a Classification Metric Wearing an Agent Costume
• The Compounding Math Nobody Puts on the Slide
• Accurate at What, Exactly?
• The Distribution Problem: Averages Hide the Failures That Hurt
• What to Measure Instead
• Task Success Rate, Defined Per Use Case
• Trajectory and Step-Level Reliability
• Cost-Weighted Error: Not All Mistakes Are Equal
• Reliability Under Repetition
• How to Read a Vendor's Accuracy Claim
• Insights Most People Overlook
• References

The Number That Sounds Great and Means Nothing

Walk any AI-agent vendor's homepage and you will eventually hit a number with a percent sign and the word "accurate" next to it. Ninety-eight percent. Ninety-nine point five. Sometimes a confident, suspiciously round "99%." It is the marketing equivalent of a firm handshake, it signals competence without committing to anything.

The problem is not that the number is fake. Often it is technically true. The problem is that "accuracy" as a concept was built for a different kind of machine, and bolting it onto an autonomous agent produces a figure that is precise, defensible, and almost entirely disconnected from whether the agent will do your job. A procurement team that buys on the strength of "99% accurate" is doing roughly what you'd do if you bought a car because the brochure said the engine fires 99% of the time. Fine, but what happens on the other 1%, how often does the trip require the engine to fire, and does the car actually reach the destination?

This is one of the load-bearing confusions in the entire agentic-AI-as-a-service market, and it deserves to be taken apart carefully.

Accuracy Is a Classification Metric Wearing an Agent Costume

Accuracy has a real, rigorous meaning. In machine learning it is the fraction of predictions a model gets right: of every example you showed it, how many did it label correctly. For a spam filter, a fraud classifier, or an image labeler, accuracy is a sensible top-line metric because the model makes one decision per input. One input, one output, one chance to be right or wrong.

An agent is not that. An agent is a loop. It reads a goal, decides on an action, calls a tool, observes the result, decides on the next action, and repeats, sometimes three times, sometimes thirty, sometimes until it gives up. Each of those decisions is a place where it can go right or wrong. So when a vendor reports "99% accurate," the immediate question is: 99% of what? Per individual tool call? Per reasoning step? Per completed task? Those are wildly different claims, and vendors are rarely eager to specify which one they mean, because the most flattering interpretation is almost always the per-step one, and the per-step number is the least useful thing you could possibly know.

There's a deeper issue. Classification accuracy assumes a ground-truth label exists for every input. For a lot of agentic work, there is no single correct answer. "Resolve this customer's billing complaint" can be done five legitimate ways. "Research our top three competitors and summarize their pricing" has no answer key. The moment the task is open-ended, accuracy stops being measurable in the classification sense at all, and the number on the slide is measuring some narrow proxy the vendor chose, usually whichever proxy looks best.

The Compounding Math Nobody Puts on the Slide

Here is the part that should change how you read every agent benchmark you ever see again.

Suppose an agent is genuinely 99% reliable at each step, a fantastic per-step number, better than most production agents achieve today. Now suppose your task takes 20 steps, which is modest for anything involving real tool use, retrieval, and multi-system coordination. The probability that all twenty steps go right is 0.99 raised to the 20th power.

That's about 0.818. Roughly 82%. Your 99%-accurate agent fails nearly one in five tasks.

Stretch the task to 50 steps, still realistic for a complex workflow like onboarding a vendor or reconciling an invoice across three systems, and 0.99^50 is about 0.605. Your headline-99% agent now completes the full job correctly only 60% of the time. Drop the per-step reliability to a more honest 95%, run it over 50 steps, and you're at 0.95^50 ≈ 0.077. Under 8%. The agent essentially never finishes the whole task right.

This is the compounding-error problem, and it is the single most important fact about agent reliability. It is also why the most useful framing of an agent's quality is not a percentage at all but a question: how long a chain of decisions can this thing sustain before the odds turn against it? Researchers studying long-horizon tasks have started measuring exactly this, the task length at which success drops below 50%, precisely because per-step accuracy tells you nothing once you know steps multiply. The analysis behind METR's work on measuring AI agents by task length makes the point concretely: capability is better expressed as the horizon an agent can handle than as a single accuracy figure.

The marketing incentive here is obvious and worth naming. Per-step accuracy is the number that survives compounding the best, because it never gets multiplied out. So that is the number you get shown.

Accurate at What, Exactly?

Even setting compounding aside, "accurate" smuggles in an undefined scope. Consider an agent that handles expense reports. It could be:

• Accurate at extracting the dollar amount from a receipt (a perception task).
• Accurate at categorizing the expense (a classification task).
• Accurate at deciding whether it violates policy (a judgment task).
• Accurate at completing the full reimbursement end to end, including filing it in the right system (a workflow task).

A vendor can legitimately quote 99% on the first one and quietly let you assume it applies to the fourth. The receipt-reading model might be excellent; the end-to-end completion rate might be 70%. Both numbers are real. Only one of them is the number you're buying. This is the same gap that separates a "capable" agent from a deployable one, a distinction the GaaS field is increasingly built around, and the subject of plenty of debate about why impressive demos so often stall before production.

When a number lacks a denominator you understand, treat it as decorative.

The Distribution Problem: Averages Hide the Failures That Hurt

Suppose you pin a vendor down and they give you an honest end-to-end task success rate: 92%. Better. But an average is still a flat summary of a shape, and the shape is where the risk lives.

Imagine two agents, both 92%. Agent A's 8% of failures are evenly spread, mostly low-stakes, and it reliably flags uncertainty before doing anything irreversible. Agent B is 92% too, but its failures cluster on exactly the high-value, edge-case transactions, and when it fails it fails silently, confidently producing wrong output with no signal that anything went wrong. These two agents have identical accuracy and completely different risk profiles. One is deployable. The other is a lawsuit waiting for a trigger.

This is why mature evaluation looks past the mean to the distribution: where do failures land, how bad are they when they land, and does the agent know when it's in trouble? The silent-failure mode, an agent that confidently does the wrong thing, or confidently does nothing useful while reporting success, is far more dangerous than a loud, obvious crash, because the loud failure gets caught and the silent one ships to your customer.

What to Measure Instead

If accuracy is the wrong lens, what's the right one? Not a single number, a small panel of them, each answering a question accuracy can't.

Task Success Rate, Defined Per Use Case

The honest top-line metric is end-to-end task success: of N complete tasks attempted, how many reached a correct, usable outcome, judged against a definition of "correct" you wrote, not the vendor. The definition is the hard part and the valuable part. "Successfully resolved the ticket" has to be operationalized into something checkable. Leading vendors who take this seriously build golden datasets of representative tasks with explicit rubrics, and they report success against that, not against a tidy classification set. If a vendor can't tell you exactly how they define a successful task, the success rate is meaningless regardless of how high it is.

Trajectory and Step-Level Reliability

Because tasks are chains, you want visibility into the chain. Trajectory-level evaluation looks at the whole path the agent took, not just the final answer, did it call the right tools in a sensible order, recover from a bad tool response, avoid wandering? Two agents can land on the same correct answer with totally different trajectories, one efficient and one that burned forty tool calls and got lucky. The lucky one will not stay lucky at scale. Anthropic's guidance on building effective agents leans on this idea: simpler, more legible trajectories are more reliable trajectories, and you can only see that if your evaluation watches the path.

Cost-Weighted Error: Not All Mistakes Are Equal

A 1% error rate means nothing until you weight each error by its consequence. In some verticals a false positive is a minor annoyance and a false negative is catastrophic; in others it's reversed. An agent that approves a fraudulent transaction and an agent that flags a legitimate one both "made an error," and treating those as interchangeable is how you ship something that's statistically excellent and operationally ruinous. The right metric multiplies error frequency by error cost, which is why the cost asymmetry of false positives versus false negatives varies so much by vertical and has to be modeled per use case, not assumed.

Reliability Under Repetition

Agents are stochastic. Run the same task twice and you can get two different outcomes, the reproducibility problem that quietly undermines a lot of single-shot benchmark numbers. A real reliability metric runs each task many times and reports the consistency, not the best run. "Pass@1 averaged over 50 trials with variance" tells you something. "We ran it once and it worked" tells you nothing. If a vendor's impressive number came from a single pass, it's a demo, not a measurement.

How to Read a Vendor's Accuracy Claim

So you're on the call and the slide says 99%. Ask, in order:

1. Ninety-nine percent of what unit, steps, tool calls, or completed tasks? If it's not completed tasks, mentally discount it hard and ask for the task-level number.
2. How do you define a successful task, and who wrote that definition? If the rubric is theirs and they won't share it, the number is unauditable.
3. Over how many steps does a typical task run, and what's the success rate at the long end of that distribution? This surfaces compounding.
4. What does the failure distribution look like, where do failures cluster, and are they loud or silent? This surfaces the risk shape.
5. Was this measured over many repetitions, and what's the variance? This surfaces reproducibility.

A vendor who answers all five crisply is a vendor who actually measures their system. A vendor who keeps steering back to the single headline number is telling you, without meaning to, that the headline number is all they've got. Industry analysts have repeatedly flagged that a large share of agentic AI projects underdeliver against expectations, and Gartner's predictions on agentic AI project cancellations point at exactly this gap between marketed capability and delivered reliability. The accuracy theater is part of how that gap stays hidden until it's expensive.

Insights Most People Overlook

The vendor incentive runs toward the most fragile metric, not the most useful one. Per-step accuracy is the only metric that doesn't get destroyed by compounding, which is precisely why it's the one you get shown. The number you see is selected because it survives the math that matters, that's not a coincidence, it's the whole point. Treat any single accuracy figure as the most flattering true statement the vendor could find.

A higher per-step accuracy can ship a worse product if it buys overconfidence. Two agents at 95% and 98% per-step: the 98% one may be more dangerous if its remaining failures are silent and the team trusts it enough to remove human review. The 95% agent that loudly escalates when unsure can have a better effective reliability in production, because its errors get caught. Raw accuracy and deployed safety can move in opposite directions, and the escalate-to-human design often matters more than the last few accuracy points.

The right reliability metric is a horizon, not a percentage. The most predictive way to describe an agent is the task length at which it crosses below 50% success. This single framing absorbs both per-step quality and chain length into one number that actually forecasts what the agent can be trusted to attempt. A vendor who reports their horizon understands their own system better than one who reports a bare percentage, and it's a far harder number to game.

"Accuracy" is sometimes literally undefined for the task, and the number is then a category error. For open-ended work with no ground-truth answer key, there is no accuracy to measure, only success against a rubric someone chose. When a vendor quotes accuracy on an open-ended task, they've quietly swapped in a proxy and not told you. The question "accurate against what reference set?" frequently has no good answer, and asking it is the fastest way to find out whether the vendor is measuring reality or marketing.

Reliability is the real moat, and accuracy theater obscures who actually has it. Capability is increasingly commoditized, everyone has access to similar frontier models. What separates a GaaS company that survives from one that gets canceled is sustained, measured, distribution-aware reliability, which is genuinely hard to build and harder to fake. The companies leaning on a single accuracy number are usually the ones who haven't done the unglamorous work of measuring the distribution, the trajectory, and the horizon. The number is loud because the substance is thin.

References

• Measuring AI Ability to Complete Long Tasks, METR
• Building Effective Agents, Anthropic
• Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027, Gartner