Long-Running Agent Execution: Why Orchestration Is the Hard Part of GaaS

Table of Contents

• The Problem Hiding Behind Every Agent Demo
• What "Long-Running" Actually Means
• Why Ordinary Request/Response Infrastructure Breaks
• Durable Execution: The Pattern That Quietly Won
• The Four Hard Problems of Orchestration
• State and Memory Across Restarts
• Idempotency and the Double-Charge Problem
• Human-in-the-Loop and Indefinite Waits
• Failure, Retries, and Knowing When to Stop
• The Build-vs-Buy Decision for Operators
• How This Reshapes GaaS Economics
• Insights Most People Overlook
• References

The Problem Hiding Behind Every Agent Demo

Watch any polished agent demo and you'll see the same trick. The agent gets a clean task, calls three or four tools, narrates its reasoning, and lands a tidy result inside a couple of minutes. It looks like magic, and it makes for a great screen recording.

Then you try to ship it.

The moment an agent has to do something that takes real time, reconcile a quarter of invoices, run a multi-day outbound research campaign, migrate a database while a human signs off at three checkpoints, the demo physics stop applying. The process that felt like a single uninterrupted thought turns out to be a long, fragile chain of operations, any one of which can fail. The model times out. The third-party API returns a 429. The server hosting your agent gets recycled by your cloud provider at 2 a.m. And suddenly the question isn't "can the model reason well?" It's "what happens to a four-hour job when the thing running it dies at minute 200?"

That question is the orchestration challenge, and it is the single most underappreciated part of building Agentic-AI-as-a-Service. Reasoning quality gets the headlines. Orchestration decides whether you have a product.

What "Long-Running" Actually Means

It helps to be precise, because "long-running" is doing a lot of work in that sentence. There are three distinct things people mean, and they fail differently.

The first is wall-clock duration. A job that runs for hours or days spans more failure events than a job that runs for seconds. Cloud instances get preempted. Deployments roll. Network blips happen. The longer the clock runs, the higher the odds that something interrupts you, and the more expensive it is to start over.

The second is step count. Some agents finish in two minutes but take four hundred LLM calls and tool invocations to get there. Each step is a chance to drift off course, exhaust a context window, or hit a tool that misbehaves. A long chain is "long-running" even if the wall clock is short, because the surface area for failure scales with steps, not seconds.

The third, the one people forget, is suspension. An agent that submits a refund request and then waits two days for a human manager to approve it is long-running in the most demanding sense. It isn't computing. It's parked, holding state, waiting for an external event that might arrive in five minutes or never. Keeping a process alive and correct while it does nothing is a surprisingly different engineering problem from keeping it alive while it works.

Most serious GaaS workloads are some blend of all three. That blend is exactly what naive infrastructure can't handle.

Why Ordinary Request/Response Infrastructure Breaks

The web was built on a beautiful assumption: requests are short and stateless. A user hits an endpoint, the server does a little work, returns a response, and forgets everything. Load balancers, autoscalers, serverless functions, and HTTP timeouts are all tuned for this rhythm. It's why the modern cloud is cheap and elastic.

Agents violate every part of that assumption.

An agent's "request" can last hours. It carries deep, evolving state, a memory of everything it's done, the partial results it's accumulated, the plan it's revising. It pauses on external events. And it has to be resumable: if it dies at step 200 of 400, you cannot just retry the whole HTTP request, because steps 1 through 199 may have sent emails, charged cards, or written to a production database. Replaying them is not a no-op. It's a disaster.

So operators reach for the obvious tools and watch each one fail in a specific way:

• Serverless functions cap out at a few minutes and have no memory between invocations. Fine for a single tool call, useless for a four-hour workflow.
• A long-lived server holding the job in RAM works right up until that server restarts, and then the entire in-memory job evaporates with no record of where it was.
• A background job queue (Celery, Sidekiq, BullMQ) gives you retries, but a blind retry re-runs the whole task from the top, re-triggering every side effect you already committed.
• A hand-rolled state machine in a database can work, but you end up reinventing checkpointing, retries, timeouts, and recovery by hand, badly, and forever.

The throughline is that all of these were designed for short, stateless, idempotent work. Agents are long, stateful, and full of irreversible side effects. The infrastructure mismatch is the whole problem.

Durable Execution: The Pattern That Quietly Won

The cleanest answer the industry has converged on comes from an unglamorous place: the world of distributed systems and workflow engines, predating the current agent wave by years. It's called durable execution, and it's the backbone of systems like Temporal, AWS Step Functions, Restate, and Inngest, now being repackaged explicitly for agents.

The core idea is deceptively simple. Every meaningful step an agent takes, every LLM call, every tool invocation, every decision, is recorded to a persistent, append-only log as it happens. The engine treats your agent's code as a deterministic function that can be re-executed. When a crash happens and the process restarts, the engine replays the code, but instead of re-running each completed step, it feeds back the recorded result from the log. The agent fast-forwards through everything it already did and resumes precisely where it left off, same memory, same partial results, same plan.

Anthropic's own guidance on building agents leans hard on this separation between the reasoning loop and the execution environment; their writeup on building effective agents is worth reading as a primer on keeping the orchestration layer simple and observable rather than letting the model improvise its own control flow.

What durable execution buys you, almost for free once it's in place:

• Crash recovery without replaying side effects. Completed steps are never re-executed; their results are read from the log.
• Suspension that costs nothing. A workflow waiting on a human approval can be evicted from memory entirely and rehydrated when the event arrives. You aren't paying to keep a server idling for two days.
• A built-in audit trail. The execution log is, conveniently, exactly the record you need for debugging, compliance, and billing, which matters enormously for outcome-priced GaaS.

This is why "durable execution for agents" has become one of the fastest-moving corners of the infrastructure stack. It's the load-bearing wall.

The Four Hard Problems of Orchestration

Durable execution is the framework, but it doesn't make the hard problems disappear. It gives you a place to solve them. Here are the four that bite every team.

State and Memory Across Restarts

An agent's state is bigger than a row in a database. It includes the conversation history, the scratchpad of intermediate reasoning, accumulated tool results, and the evolving plan. All of this has to be externalized, written somewhere durable, rather than living only in the process's memory.

The subtlety is what to persist and when. Persist too little and a restart loses critical context. Persist the entire context window after every step and your storage and serialization costs explode, especially on long jobs. Most mature systems separate durable workflow state (the log of what happened) from the agent's working memory (what it currently needs in context), and treat them as related but distinct concerns. This is where long-running execution bleeds directly into state management for stateful agents and the broader question of managing what agents remember as context windows fill and token budgets tighten.

Idempotency and the Double-Charge Problem

This is the one that ruins production weekends. Imagine an agent that, at step 150, charges a customer $400 through a payment API. The charge succeeds, but the agent crashes before recording that success to its log. On restart, the engine replays, reaches step 150, and charges the customer again.

The defense is idempotency: every side-effecting operation must be designed so that running it twice has the same effect as running it once. In practice that means attaching idempotency keys to external calls (so the payment provider deduplicates the second attempt), wrapping side effects in the durable engine's "activity" or "step" primitive so their results are logged atomically, and assuming every tool call might be retried. This is unglamorous discipline, and it's the difference between an agent that's safe to point at real systems and one that isn't. It connects tightly to tool-calling reliability at the infrastructure layer and the patterns behind retries, fallbacks, and circuit breakers.

Human-in-the-Loop and Indefinite Waits

A genuinely autonomous agent still needs humans at the high-stakes moments, approving a wire, signing off on a contract, reviewing generated code before it merges. From the orchestration layer's view, a human approval is just an external event with an unbounded arrival time. Could be thirty seconds. Could be a week. Could be never, if the approver quits.

Handling this well means the workflow must suspend durably, hold its full state, expose a callback or signal the outside world can fire, and enforce a timeout with a sensible fallback (escalate? cancel? proceed with a default?). Teams that bolt human checkpoints on as an afterthought discover their agents either spin a billable server while waiting or lose state when the wait outlasts the process. Designing for human-in-the-loop checkpoints from the start is far cheaper than retrofitting them.

Failure, Retries, and Knowing When to Stop

Not all failures are equal, and treating them the same is a classic mistake. A transient 429 rate-limit should be retried with exponential backoff. A malformed-input error will fail identically forever, so retrying it just wastes money and time. And then there's the failure mode unique to agents: the model that confidently loops, re-trying the same broken approach, burning tokens, convinced it's making progress.

A real orchestration layer classifies errors, applies backoff selectively, sets hard ceilings on retries and on total steps and total spend, and has a clean escalation path when an agent is stuck rather than letting it grind indefinitely. The budget ceiling is not optional. An unsupervised agent with a credit card and a retry loop is a way to generate a very expensive incident report.

The Build-vs-Buy Decision for Operators

If you're running a GaaS business rather than writing a distributed-systems thesis, the practical question is: do you build the orchestration layer or adopt one?

Building it yourself means owning checkpointing, replay, idempotency, timeout handling, and recovery, a deep, subtle body of work that has very little to do with your actual product. The teams I've watched go this route almost always underestimate the long tail: the recovery is the easy 80%, and the gnarly 20% (partial failures, replay determinism, race conditions around side effects) is where the months go.

Adopting a durable-execution engine, Temporal, Inngest, Restate, AWS Step Functions, or one of the newer agent-native runtimes, means inheriting battle-tested machinery for exactly these problems, at the cost of a dependency and a learning curve. For most operators this is the right call, and it's part of why the agent runtime is emerging as a distinct infrastructure category. The leverage is enormous: you spend your engineering on the agent's actual behavior and your domain, not on reinventing crash recovery.

The deeper point is strategic. McKinsey's analysis of the shift toward agentic AI in the enterprise frames the move from copilots to autonomous agents as fundamentally an orchestration and integration challenge, not a model-quality one, the bottleneck is wiring agents reliably into real systems and workflows, which is precisely the long-running execution problem under a different name.

How This Reshapes GaaS Economics

Here's why an operator should care about all of this beyond engineering aesthetics: reliable long-running execution is what makes outcome-based pricing possible.

If you want to charge per completed task or per successful outcome, the pricing model that distinguishes true GaaS from a thin wrapper over an API, you have to be able to guarantee completion. A job that silently dies halfway, or double-charges, or stalls forever waiting on an event nobody fires, is a job you can't bill for and a customer you might refund. Outcome pricing only works on top of execution you trust.

Durable execution also gives you the data to price intelligently. Because every step is logged, you know exactly what a given outcome cost in tokens, tool calls, and compute, which feeds directly into the GaaS infrastructure cost stack and lets you set margins instead of guessing at them. And the same log that powers recovery powers the audit trail enterprise buyers demand before they'll let an autonomous agent touch their systems.

So orchestration isn't a back-office concern. It's the thing standing between "interesting demo" and "a service people will pay outcomes for." The companies that win the GaaS market won't necessarily have the smartest agents. They'll have the agents that finish.

Insights Most People Overlook

1. The bottleneck has quietly moved from the model to the runtime. A year ago the limiting factor in agent quality was reasoning. For most production workloads now, the model is good enough, and the thing that determines whether the agent works is the orchestration layer's ability to survive failure. Teams keep investing in better prompts and bigger models when the actual leak is in their execution infrastructure. The smartest agent in the world is worthless if it loses its place every time a server recycles.

2. Determinism is the hidden tax on durable execution, and it surprises everyone. Replay-based engines require your workflow code to be deterministic, same inputs, same path, every time. But LLMs are non-deterministic by nature, and so is random(), now(), and any unguarded external read. The discipline of cordoning all non-determinism behind logged "activity" boundaries is non-obvious and trips up nearly every team's first durable agent. People expect the hard part to be the AI. The hard part is making the AI's surroundings boringly predictable.

3. Indefinite suspension is harder than crash recovery, and gets the least attention. Everyone designs for "what if it crashes." Far fewer design for "what if it waits a week." A workflow parked on a human approval has to hold state correctly while contributing zero compute, survive deployments during the wait, and handle the approver who never responds. This passive-but-alive state is its own engineering discipline, and it's where naive human-in-the-loop implementations quietly rot.

4. Your recovery log is also your best observability and billing artifact, most teams discover this by accident. Operators build durable execution for reliability, then realize the append-only step log is exactly the substrate they need for debugging "what did the agent actually do," for the compliance audit trail enterprise deals require, and for per-outcome cost accounting. If you design the log with those uses in mind from day one rather than retrofitting them, you get three features for the price of one.

5. "Don't make the agent orchestrate itself" is counterintuitive but usually correct. There's a temptation to let the LLM manage its own control flow, deciding when to retry, when to checkpoint, when to wait. Resist it. The reliable pattern is a dumb, deterministic orchestration layer wrapping a smart, stochastic reasoning core. The model decides what to do; the runtime decides how to do it durably. Blurring that line is how you get agents that are simultaneously brilliant and unshippable.

References

• Building Effective Agents, Anthropic Engineering
• Why Agents Are the Next Frontier of Generative AI, McKinsey & Company