Self-Hosted vs. Managed Agent Infrastructure: The Build-or-Buy Decision That Actually Matters

Table of Contents

• Why This Decision Looks Different for Agents
• What "Managed Agent Infrastructure" Actually Includes
• What You Take On When You Self-Host
• The Cost Curve Nobody Models Correctly
• Decision Factors That Should Drive the Call
• Data residency and compliance
• Workload shape and concurrency
• Team maturity and on-call appetite
• The Hybrid Pattern Most Mature Teams Land On
• Insights Most People Overlook
• References

Why This Decision Looks Different for Agents

Build-or-buy is an old infrastructure debate. We had it about databases, about Kubernetes, about message queues. So it's tempting to treat agent hosting as just another instance of the same trade-off. It isn't, quite.

A traditional web service is mostly stateless request-response. An agent is a long-running, stateful, failure-prone loop that calls a model, calls tools, waits, retries, and sometimes hangs for minutes or hours. The infrastructure has to hold conversation state and memory, manage a context window that grows and gets expensive, sandbox tool execution so a hallucinated shell command doesn't wipe a volume, and survive a model provider returning a 529 halfway through a thirty-step workflow. None of that maps cleanly onto the autoscaling-stateless-pods playbook that cloud-native infra was built around.

That's why the "agent runtime" has emerged as its own infrastructure category rather than a thin wrapper over existing compute. And it's why the self-hosted-versus-managed decision carries more weight here: you're not choosing where to run a simple service, you're choosing who owns a genuinely hard distributed-systems problem. In the GaaS world, where agents are sold per-task or per-outcome, that ownership question bleeds straight into your unit economics. The infrastructure you pick determines your cost of goods sold.

What "Managed Agent Infrastructure" Actually Includes

When a vendor sells you a managed agent platform, the pitch is "deploy an agent, we handle the rest." The "rest" is doing a lot of work, and it's worth itemizing because the value is real even if the marketing is fuzzy.

A serious managed platform gives you durable execution, so a workflow that crashes at step 19 resumes at step 19 instead of restarting and re-billing nineteen model calls. It gives you a state and memory layer you didn't have to architect. It gives you the observability stack: traces across model calls and tool calls, token accounting, latency breakdowns, replay. It handles concurrency and queueing so a thousand simultaneous agent runs don't melt your model rate limits. It usually bundles tool sandboxing and some flavor of identity and policy enforcement at the gateway.

Reproducing that list internally is months of senior-engineer time. The managed pitch is that you skip the months. Providers like LangGraph Platform, AWS Bedrock AgentCore, and a wave of agent-runtime startups are all selling variations of this bundle. The convenience is genuine. Independent surveys keep showing the same gap between agent ambition and production reality, and Andreessen Horowitz's writing on the emerging agent infrastructure stack lays out why so much net-new tooling is being built specifically because the old stack doesn't fit.

The catch is threefold. You pay a margin on top of raw inference. You inherit the platform's opinions about how agents should be structured, which is fine until your use case fights the abstraction. And you're exposed to their roadmap, their outages, and their pricing changes, with limited recourse. Lock-in here isn't just data gravity, it's execution-model gravity. Rewriting agents off a proprietary orchestration framework is not a weekend migration.

What You Take On When You Self-Host

Self-hosting agent infrastructure means standing up the orchestration layer, the state store, the queue, the sandbox, the observability pipeline, and the scaling logic yourself, usually by assembling open-source pieces. A common stack is a durable-execution engine like Temporal under an open framework like LangGraph or a homegrown loop, a vector store and a memory layer for retrieval, a container sandbox for tool calls, and an OpenTelemetry-based trace pipeline feeding something like Grafana.

The upside is total control. You see every cost line, so you can route to a cheaper model when the task allows, cache aggressively, and trim token budgets at runtime without a vendor's pricing model fighting you. You decide where data lives, which matters enormously if you're in healthcare, finance, or any regime with hard residency rules. You can run inference on your own GPUs or against whichever provider is cheapest this quarter, avoiding the vendor lock-in that comes from building on one platform's primitives. And when something breaks, you can actually fix it instead of filing a support ticket and waiting.

The cost is everything you'd expect and a few things you wouldn't. You own on-call. You own the security posture of a system that executes model-generated code, which is a meaningfully larger attack surface than a normal app. You own the long tail of reliability engineering, retries, fallbacks, circuit breakers, the dead-letter queue for runs that wedge. And you own the opportunity cost: every week your best engineers spend on agent plumbing is a week they don't spend on the product. Plenty of teams underestimate this and end up having quietly rebuilt a worse version of a managed platform, eighteen months later, at higher total cost.

The Cost Curve Nobody Models Correctly

The cost comparison gets done badly almost universally, because people compare the wrong things. The naive version is "managed charges a 30% markup, self-hosting is just the raw inference, therefore self-hosting is cheaper." That ignores the entire denominator.

Here's the more honest model. Managed cost is roughly inference plus platform fee, and it scales smoothly with usage with near-zero fixed overhead. Self-hosted cost is inference plus infrastructure plus the fully loaded engineering cost to build and operate the platform, and that engineering cost is mostly fixed. So the two curves cross. Below some volume, the managed platform's per-unit margin is cheaper than amortizing your engineering team across a small workload. Above it, your fixed costs amortize away and self-hosting wins on the margin.

The crossover point is much higher than engineers want to believe, because they systematically undercount the operational tail: the on-call rotation, the incident response, the security audits, the upgrade treadmill as model providers and frameworks change under you. McKinsey's analysis of the economic potential and cost realities of generative AI is a useful corrective here: the dominant cost in most production deployments is people and integration, not the model bill. The inference line you're so focused on optimizing is often not where your money actually goes.

For a GaaS operator the calculus has an extra twist. If you sell per-outcome, your infrastructure cost is your margin, and at high volume a few cents per task compounds into real money. That's the case where self-hosting's marginal advantage justifies the fixed investment, and it's exactly why mature, high-volume agent businesses tend to repatriate. But repatriating prematurely, before you have the volume or the reliability discipline, is how startups light cash on fire.

Decision Factors That Should Drive the Call

Strip away the ideology and a few concrete factors should dominate the decision.

Data residency and compliance

If you're handling regulated data with hard residency or sovereignty requirements, the question may already be answered for you. Some managed platforms offer dedicated regions, BYO-cloud deployments, or in-VPC options that satisfy auditors, and those are worth seeking out before defaulting to self-host. But if you need model-generated outputs and intermediate agent state to never leave a specific boundary, and no managed vendor can contractually guarantee it, self-hosting stops being a cost question and becomes a requirement. Don't let an engineer's preference make a compliance decision.

Workload shape and concurrency

Spiky, unpredictable workloads favor managed, because elastic scaling is exactly the thing that's painful to build and the thing platforms do well. Steady, high, predictable volume favors self-hosting, because you can right-size reserved capacity and stop paying someone else's autoscaling premium. Long-running agents, the ones that execute for minutes or hours, stress durable execution and state management hard enough that you want either a mature managed platform or a serious durable-execution engine like Temporal underneath, never a hand-rolled loop. If your agents are short and stateless-ish, your infrastructure needs are lighter and the managed premium buys you less.

Team maturity and on-call appetite

Be honest about who's going to carry the pager. A team without distributed-systems depth that self-hosts agent infrastructure is signing up to learn durable execution, sandboxing, and observability in production, under incident pressure. That's a brutal way to learn. A team that already runs Temporal, Kubernetes, and a real observability stack can absorb agent infra as an extension of what they do. The platform you can operate reliably beats the platform that's theoretically cheaper but you can't keep up at 3 a.m.

The Hybrid Pattern Most Mature Teams Land On

The framing as a binary is the real mistake. In practice the destination for most serious agent operators is a hybrid, and it tends to evolve in a predictable arc.

Start managed. You're searching for product-market fit and you cannot afford to spend your runway building orchestration plumbing nobody will ever see. The managed premium is cheap relative to the speed it buys. Ship agents, learn what they actually do in production, find out which workflows matter.

Then, as volume concentrates, repatriate selectively. You'll usually find that a small number of high-volume, well-understood, stable workflows account for most of your inference spend. Those are the candidates to move to self-hosted infrastructure where the per-task economics justify the operational burden. The long tail of low-volume, experimental, or fast-changing agents stays managed, because the engineering cost to self-host them would never pay back.

This is the same repatriation pattern that played out in cloud computing broadly, and the a16z analysis of the cost of cloud and the repatriation trend is the canonical articulation of it: rent until your usage is large and stable enough that owning is cheaper, then own the predictable core and keep renting the variable edge. Agents are following the same logic, just faster, because the cost pressure of per-token inference makes the math sharper. A clean abstraction layer over your orchestration, so the agents themselves don't hard-code which backend runs them, is what makes this arc possible without painful rewrites. Build that boundary early even if you never use it, because the day you want to move a workload, you'll be very glad it's there.

Insights Most People Overlook

The real lock-in is the execution model, not the data. Everyone worries about data gravity, but the thing that actually traps you in a managed agent platform is its opinion about how agents are structured, how state is shaped, how tools are defined. You can export your data in an afternoon. Rewriting agents off a proprietary orchestration abstraction can take a quarter. Evaluate the exit cost of the framework, not just the data.

Self-hosting doesn't reduce your model-provider dependency, and people conflate the two. Pulling your orchestration in-house feels like independence, but unless you're also running open-weight models on your own GPUs, you're still entirely exposed to your inference provider's pricing, rate limits, and outages. Most "self-hosted" agent stacks are self-hosted orchestration calling a managed model API. That's a partial hedge, not sovereignty, and the GPU-supply constraint means true model self-hosting at scale is its own expensive project.

The security delta is larger than the cost delta and gets discussed less. An agent that executes model-generated tool calls is a fundamentally different threat model from a normal app. When you self-host, you own sandboxing, prompt-injection defenses, and the blast radius of an agent that gets manipulated into doing something it shouldn't. Managed platforms aren't automatically safe, but at least someone whose full-time job is this is thinking about it. Teams that self-host to save money frequently underinvest here and find out the hard way.

Durable execution is the dividing line between a demo and a product, and it's the hardest part to build. The single most underestimated piece of agent infrastructure is making a long, multi-step workflow survive crashes and resume without re-running expensive steps. It's also the piece teams most often skip when they roll their own, which is why so many self-hosted agents work fine in the demo and fall apart under real load. If you're going to self-host, decide on your durable-execution strategy first, not last.

Premature repatriation kills more agent projects than vendor lock-in does. The fear of lock-in pushes teams to self-host before they have the volume to justify it, and they spend their scarce engineering capacity building infrastructure instead of finding out whether anyone wants the product. Lock-in is a problem you get to have once you're successful. Most projects die long before that.

References

• The Agentic Web and Its Infrastructure (Andreessen Horowitz)
• The Cost of Cloud, a Trillion Dollar Paradox (Andreessen Horowitz)
• The Economic Potential of Generative AI: The Next Productivity Frontier (McKinsey)