How Tool-Call Costs Stack and Compound in Agent Workflows

Table of Contents

• The Mental Model Most People Get Wrong
• Anatomy of a Single Tool Call
• Why Costs Compound Instead of Adding Up
• Worked Example: A 12-Step Research Agent
• The Four Cost Multipliers Nobody Budgets For
• Levers That Actually Bend the Curve
• How This Fits the GaaS Economics Picture
• Insights Most People Overlook
• References

The Mental Model Most People Get Wrong

Ask a founder what a tool call costs and you'll usually get an answer about the tool itself. The search API is a tenth of a cent. The database query is free. The Slack post is free. So tool calls are basically free, right?

That accounting misses where the money is. In an agentic workflow, the expensive part of a tool call is almost never the tool. It's the model round-trip wrapped around it: the tokens the model spends deciding to call the tool, plus the tokens it spends reading the result back on the next turn, plus the tokens that result keeps costing on every turn after that because it's now sitting in the conversation history.

I've watched teams obsess over shaving a few cents off a vendor API while their model bill for the same workflow ran 40x higher. The tool fee is the rounding error. The inference around it is the COGS.

Once you internalize that, the whole cost structure of an agent looks different. You stop thinking of a workflow as "ten tool calls at a tenth of a cent each" and start thinking of it as "ten model invocations whose prompts keep getting longer." That second framing is the one that predicts your invoice.

Anatomy of a Single Tool Call

Strip a tool call down to its parts and you can see the cost surface. Each call has four token components, and only one of them is the obvious one:

1. The decision tokens. The model reads the current context and generates a structured tool call. That output is billed at output rates, which on frontier models run several times the input rate. If your agent "thinks" before each call (and most reasoning-tuned models do), the reasoning trace is billed too, even though you never see it.

2. The tool schema overhead. Every tool you expose lives in the system prompt as a JSON schema. Twenty tools with rich descriptions can easily be 4,000-8,000 tokens of input that the model re-ingests on every single turn of the conversation, whether or not it uses any of them. Most teams never measure this, and it's a flat tax on every step.

3. The result payload. The tool returns something, a search result set, a 200-row query, a webpage's text, and that payload gets injected back into context as input tokens for the next model call. A verbose tool that returns raw HTML or unfiltered JSON can dump 10,000+ tokens into your prompt in one shot.

4. The persistence tax. Here's the one that compounds. That result payload doesn't go away after the model reads it once. Unless you actively prune or summarize, it stays in the conversation history and gets re-sent, and re-billed as input, on every subsequent turn.

Anthropic's own guidance on building effective agents leans hard on keeping context lean precisely because of this dynamic: the agent loop re-reads its accumulated state constantly, and unmanaged state is the silent budget killer.

Why Costs Compound Instead of Adding Up

This is the heart of it, and it's worth being precise.

Imagine an agent that runs N steps. At each step it calls a tool, gets a result of roughly R tokens, and that result sticks around in context. By the linear intuition, total cost is N tool calls, a straight line. But that's not what the model is billed for.

At step 1, the model processes the base prompt. At step 2, it processes the base prompt plus step 1's result. At step 3, base plus results 1 and 2. By step N, the model is re-reading every prior result it accumulated. The input tokens processed across the whole run aren't N×R, they're closer to R×(1 + 2 + 3 + ... + N), which is R×N(N+1)/2. That's quadratic.

Double the length of a task and you don't double its cost. You roughly quadruple the context-driven portion of it. This is the single most important fact about agent economics that doesn't show up in a per-call price sheet, and it's why the hidden cost of retries, where one task balloons into fifty model calls, is so brutal: every retry doesn't just add a call, it adds a call running against the longest, most expensive version of the context.

The compounding has a second source too: fan-out. When an agent spawns sub-agents, each sub-agent runs its own loop with its own growing context, and the parent has to read their summarized outputs back. A workflow that looks like "one task" on the invoice line can be a tree of dozens of independent quadratic curves. Fan-out economics deserve their own treatment, but the mechanism is the same, context that gets re-read is context that gets re-billed.

Worked Example: A 12-Step Research Agent

Let me make this concrete with numbers. Take a mid-complexity research agent: it searches, reads sources, queries a database, and synthesizes an answer over 12 tool-using steps. Round figures, frontier-class model, input at $3 per million tokens and output at $15 per million.

• Base system prompt + tool schemas: 6,000 tokens, re-sent every step.
• Average tool result: 3,000 tokens, accumulating in context.
• Average model output per step: 800 tokens (including some reasoning).

Step 1 processes ~6,800 input tokens. Step 12 processes the 6,000 base, plus eleven accumulated results at 3,000 each (33,000), plus its own working context, call it ~40,000 input tokens. The input cost per step at step 12 is nearly six times what it was at step 1, even though it's "the same kind of step."

Sum it up: the accumulated input across 12 steps lands around 280,000 input tokens, not the 80,000 you'd guess if each step were independent. At $3 per million that's about $0.84 in input alone, plus roughly $0.14 in output, for a single task. The tool APIs themselves? Maybe a cent, all in.

So 98% of the cost of this "tool-heavy" agent is the model re-reading its own history. If you'd budgeted this task by counting tool calls, you'd have underpriced it by an order of magnitude. This is exactly the trap behind the assumption that you can just pass through model costs and call it pricing, the pass-through math only works if you can predict the context curve, and most teams can't.

The Four Cost Multipliers Nobody Budgets For

Beyond the base compounding, four factors push the curve higher, and they tend to stack:

Retries and self-correction. When a tool errors or returns something the model can't use, a well-built agent retries, often after reasoning about what went wrong. Each retry runs against the full accumulated context, so failed work is the most expensive work in the run. An agent with a 30% tool-failure rate isn't 30% more expensive; it can be double, because retries hit at the fat end of the curve.

Reasoning tokens. Reasoning-tuned models generate large internal traces before acting. These are billed as output (the expensive rate) and you usually can't see them to optimize them. On a tool-heavy task, thinking-token spend can rival or exceed the visible answer. The true cost of thinking tokens is one of the least-tracked lines in agent budgets.

Oversized tool outputs. A tool that returns raw, unfiltered data is a context bomb. One un-paginated database query or one full webpage scrape can add more tokens than a dozen well-designed calls. The fix is upstream of the model, shape the tool's output, don't hope the model ignores the noise.

Schema bloat from too many tools. Every tool you register taxes every turn. Past a certain point, adding tools makes the agent both slower to choose correctly and more expensive per step. There's a real argument that a focused 8-tool agent beats a 30-tool one on both accuracy and cost.

McKinsey's analysis of the economic potential of generative AI frames agents as a productivity unlock, but the unlock only holds if cost-to-serve stays below the value delivered, and these four multipliers are precisely what erode that margin in production.

Levers That Actually Bend the Curve

The good news is that quadratic compounding is also the thing you have the most leverage over. The levers, roughly in order of impact:

Context pruning and summarization. This is the big one. Don't carry raw tool results forward forever. After a result has served its purpose, replace it with a short summary or drop it. Converting a 3,000-token search payload into a 200-token distilled note flattens the back half of the quadratic curve dramatically. The catch: summarization itself costs a model call, so it pays off on long runs, not short ones.

Prompt caching. If your base system prompt and tool schemas are stable, caching them means you stop paying full input rate to re-process those 6,000 tokens every step. On long agent loops this is one of the highest-ROI changes available, and it's nearly free to implement. Provider caching mechanics differ, so read the docs for your model.

Output shaping at the tool layer. Paginate. Filter. Return IDs and summaries instead of full records, and let the agent ask for detail only when needed. The cheapest token is the one the tool never put in context.

Sub-agent isolation. Counterintuitively, spinning up a sub-agent with a fresh, small context to handle a noisy subtask, then returning only its summary to the parent, can be cheaper than doing the subtask inline and polluting the main context for the rest of the run. You trade an extra agent for a shorter main curve.

Model routing by step. Not every step needs the frontier model. Tool-selection and simple synthesis steps can often run on a cheaper, faster model, reserving the expensive one for the steps that truly need it. The blended rate this produces is its own analysis, but the principle is simple: match the model to the marginal difficulty of the step.

How This Fits the GaaS Economics Picture

Tool-call compounding isn't a niche engineering concern. It's the mechanism underneath most of the unit-economics questions in Agentic AI-as-a-Service. When people ask why a coding agent's unit economics look ugly at scale, or why per-task pricing makes forecasting so hard, the quadratic context curve is usually lurking in the answer.

It's also why "cost-per-completed-task", the unit metric this whole category is converging on, has to be measured empirically rather than estimated from a price sheet. The same nominal task can cost 5x more on a bad run than a good one purely because of how context accumulated. Any vendor quoting you a flat per-task cost is either smoothing across a distribution or capping autonomy somewhere to keep the curve from running away.

For operators, the practical takeaway is to instrument the curve, not just the total. Track tokens-per-step, not only tokens-per-task. The shape of that per-step line tells you whether you have a pruning problem, a retry problem, or a tool-output problem long before the monthly bill does.

Insights Most People Overlook

The expensive failures are the late ones. Conventional wisdom says reduce your error rate. The sharper version: reduce errors late in the run specifically. An error at step 2 costs almost nothing to retry; the identical error at step 11 retries against 40,000 tokens of context. Two agents with the same overall success rate can have wildly different bills depending on where in the loop their failures cluster.

Cheaper tokens can make your agent more expensive. When per-token prices fall, teams loosen their context discipline, they stop pruning, they expose more tools, they let runs go longer. Because cost compounds quadratically, those looser habits can outrun the price drop entirely. Falling token prices have repeatedly failed to lower agent bills for exactly this reason.

Tool count is an accuracy and cost problem at once, and they reinforce. More tools means a bigger schema (more cost per turn) and a harder selection problem (more wrong calls, more retries, longer runs). These compound on each other, so trimming an agent's toolset often improves margin and reliability simultaneously, a rare two-for-one.

Summarization has a break-even point, and short tasks fall below it. Aggressive context-pruning is gospel advice, but it costs a model call each time. On a 3-step task, you'll spend more summarizing than you save. The right policy is length-aware: cheap accumulation early, aggressive pruning only once the curve gets steep. Teams that apply one fixed policy everywhere leave money on the table at both ends.

"Idle" context is still billed context. An agent that pauses waiting on a human or a long-running tool still holds its full accumulated context, and the moment it resumes, that whole history gets re-read at input rates. Long-running and human-in-the-loop agents pay a re-entry tax every time they wake up, a cost that's invisible if you only look at active compute time.

References

• Building Effective Agents, Anthropic
• The Economic Potential of Generative AI: The Next Productivity Frontier, McKinsey