When Compute Is Your COGS: Lessons GaaS Should Steal From Cloud

Table of Contents

• The Category Error at the Heart of GaaS Pricing
• Lesson 1: Commit-and-Burst Beats Pure On-Demand
• Lesson 2: Utilization Is the Metric, Not Revenue
• Lesson 3: Cost Attribution Has to Reach the Customer Level
• Lesson 4: Tier Your Margins, Don't Average Them
• Lesson 5: Build the FinOps Muscle Before You Need It
• Where the Cloud Analogy Breaks Down
• Insights Most People Overlook
• References

The Category Error at the Heart of GaaS Pricing

Walk into most agent startups in 2026 and you'll find a pricing page that looks like SaaS, tidy tiers, a per-seat or per-task number, maybe a "contact us" enterprise plan, bolted on top of a cost structure that behaves like a cloud bill. That mismatch is the original sin of the category.

In classic SaaS, your cost of goods sold is rounding-error small. You provision a database, you serve a few thousand API calls per user per month, and the marginal cost of the next customer is close to zero. That's why software companies report 75-85% gross margins and why investors learned to pay 10-20x revenue for them. The whole financial culture of SaaS, annual contracts, net revenue retention, the rule of 40, assumes that revenue and cost are essentially decoupled.

Agents break that assumption completely. When an agent runs, it burns inference. It calls tools. It retries. It spawns sub-agents. Every one of those actions is a metered API call against a frontier model or a GPU you're renting by the second. Your COGS isn't a flat hosting line anymore; it scales with usage, and it scales nonlinearly because of the hidden cost of retries: when one task becomes fifty model calls. The closest historical analog isn't Salesforce. It's Amazon Web Services circa 2010, a business where compute is the product, where capacity has to be procured ahead of demand, and where a few points of utilization decide whether you're profitable.

The good news: the hyperscalers already solved most of these problems. They just solved them for themselves, and GaaS founders mostly haven't bothered to read the manual. What follows is that manual, translated.

Lesson 1: Commit-and-Burst Beats Pure On-Demand

The first thing every cloud provider figured out is that on-demand pricing is a trap for the buyer and a gift for the seller, and that the smart buyer flips it by committing.

AWS, Azure, and GCP all sell the same compute three ways: on-demand (most expensive, fully flexible), reserved or committed-use (cheaper, you promise volume), and spot/preemptible (cheapest, can be reclaimed). The spread between on-demand and a one-year commitment is routinely 30-50%, and three-year commits go deeper. This isn't generosity. It's the provider buying demand certainty, which lets them plan their own capacity.

Here's the lesson GaaS keeps missing: an agent business is simultaneously a seller of agent-tasks and a buyer of inference. On the buy side, you have the same on-demand-versus-committed choice with your model providers. OpenAI, Anthropic, and Google all offer provisioned throughput, batch discounts, and committed-spend tiers that meaningfully undercut pay-as-you-go rates. If your COGS is inference and you're paying rack-rate on-demand pricing for predictable baseline volume, you're leaving 20-40 points of gross margin on the table, the exact dynamic explored in reserved vs. on-demand compute and the GaaS margin spread.

The discipline is to decompose your demand into a predictable floor and a volatile peak. Cover the floor with committed capacity. Burst the peak on-demand. Cloud capacity planners have done this with reserved instances for over a decade, and the math is identical when your "instances" are tokens per minute against a model endpoint. The mistake is treating all your inference as on-demand because that's how you started, which made sense at ten customers and is quietly bleeding you at ten thousand.

Lesson 2: Utilization Is the Metric, Not Revenue

Ask a cloud infrastructure operator what keeps them up at night and they won't say revenue. They'll say utilization. A GPU sitting idle still costs you its full hourly rate; a reserved instance you committed to but aren't using is pure loss. The entire economics of running infrastructure come down to keeping expensive, perishable capacity busy.

GaaS has the exact same problem and almost nobody is measuring it. When you provision throughput or reserve capacity to guarantee latency, every minute that capacity sits idle between agent runs is margin you've already paid for and thrown away. This is the quiet killer behind the "idle agent" cost problem and how vendors hide it, and it's structurally identical to a half-empty data center.

Steal the cloud KPI directly: track capacity utilization, not just task volume. If you've reserved enough throughput to handle peak load and your average load is 35% of peak, you're running a data center at 35% utilization, and no cloud CFO would tolerate that for long. The fixes are the same ones the hyperscalers use: smooth demand with batching and queuing, shift non-urgent work to off-peak windows, multiplex multiple customers onto shared reserved capacity, and, when you genuinely can't fill it, shed the commitment and burst on-demand instead. Google's own Site Reliability Engineering book treats utilization and headroom as first-class operational concerns, and the same rigor belongs in any agent operator's dashboard.

The deeper point is cultural. SaaS teams obsess over MRR because that's their leading indicator. Infrastructure teams obsess over utilization because that's theirs. GaaS founders who keep staring at task counts while ignoring utilization are reading the wrong gauge.

Lesson 3: Cost Attribution Has to Reach the Customer Level

In the early days of cloud, companies got their AWS bill, saw one terrifying number, and had no idea which team or product caused it. The entire FinOps discipline grew up to answer that question. Tagging, cost allocation, showback and chargeback, these exist because you cannot manage a variable cost you can't attribute.

GaaS is living through that same painful adolescence right now. Most agent vendors can tell you their total monthly inference spend. Far fewer can tell you, per customer, the fully-loaded cost to serve. And without that, you are flying blind on the single most important fact about your business: which customers are profitable and which are quietly underwater. A customer running long, retry-heavy, sub-agent-spawning workflows can cost 50x what a light user costs while paying the same per-task price, and you won't know until you measure it. That's the whole problem behind cost attribution: charging the right customer for shared agent infrastructure.

The cloud playbook is mature and directly portable. The FinOps Foundation framework lays out exactly how to attribute shared, variable cloud spend down to the team, product, and unit level, and an agent run is just another taggable unit of consumption. Instrument every agent invocation with the customer ID, the model used, token counts in and out, tool-call costs, and retry counts. Roll that up into a cost-to-serve per customer. The moment you can see that number, half your pricing and packaging decisions make themselves.

This is also the foundation for everything downstream: you cannot build a GaaS metrics dashboard every operator should track without per-customer cost attribution underneath it. It's the plumbing the rest of the financial picture sits on.

Lesson 4: Tier Your Margins, Don't Average Them

Cloud providers don't earn the same margin on everything, and they don't pretend to. Raw compute is a low-margin, competitive commodity. Managed services, a hosted database, a vector store, a serverless function, carry much fatter margins because the customer is paying for abstraction, reliability, and not having to think. AWS's profitability has long been driven less by EC2 instances than by the higher-margin managed layers stacked on top.

The lesson for GaaS is to stop reporting a single blended gross margin and start thinking in layers. The raw inference pass-through is your commodity floor; if you're competing there, you're competing with the model providers themselves and you will lose. The margin lives in everything you add on top: orchestration, reliability guarantees, memory, evaluation, the guardrails that make an autonomous agent safe to deploy. That's the value customers can't easily replicate, and it's where pricing power lives, which is also why "we'll just pass through model costs" is such a margin trap. Pass-through pricing surrenders the only layer where you make money.

Practically, this means designing your packaging so the high-margin capabilities are bundled with, not separated from, the commodity compute. Charge for the outcome and the reliability, not the tokens. The hyperscalers learned to sell DynamoDB at a premium rather than charging customers for the EC2 instances underneath it. GaaS vendors need the same instinct: the customer should be buying a completed, reliable task, while the token cost stays your internal COGS to optimize, never your headline SKU.

Lesson 5: Build the FinOps Muscle Before You Need It

The cloud era produced an entire job function, FinOps, that didn't exist before, because variable infrastructure spend turned out to need continuous, cross-functional management. Engineering, finance, and product all touch the cost, so somebody has to sit in the middle and own the optimization loop. Companies that built this muscle early controlled their cloud spend. Companies that waited until the bill scared the board ended up doing painful emergency migrations.

GaaS vendors are roughly where cloud-native companies were in 2015, spend is climbing, finance is nervous, and nobody owns it end to end. The fix is to stand up a lightweight version of the same discipline now: someone who watches inference spend daily, models the unit economics, runs the commit-versus-on-demand decisions, and feeds cost reality back into pricing. McKinsey's research on cloud cost optimization found that a large share of enterprise cloud spend is simply wasted on idle, over-provisioned, or mis-attributed resources, and the same waste categories exist, unmeasured, inside most agent platforms today.

The most underrated cloud lesson here is anomaly detection. Cloud teams live in fear of the runaway process that racks up a five-figure bill overnight, so they build alerting that catches it in minutes. Agents make this risk worse, not better, because an autonomous agent in a retry loop can burn budget faster than any human-driven process. Building a cost-anomaly alerting system for agent spend isn't a nice-to-have; it's the agent equivalent of a cloud budget alarm, and you want it before the incident, not after.

Where the Cloud Analogy Breaks Down

It would be dishonest to pretend the mapping is perfect, and the places it breaks are exactly where GaaS economics get genuinely novel.

First, your COGS isn't fully under your control. A cloud provider owns its data centers and can plan capacity on its own timeline. A GaaS vendor's primary input cost, frontier model inference, is set by a handful of suppliers who can change prices, deprecate models, or alter rate limits with weeks of notice. That introduces the token-volatility problem: budgeting when inference costs swing weekly, which has no clean cloud analog. Reserved-instance pricing was stable for years; model pricing is not.

Second, the unit of consumption is non-deterministic. A cloud VM that runs for an hour costs a known amount. An agent task that "runs once" might cost a dollar or fifty dollars depending on how many times it retries, how many tools it calls, and how much it reasons, and you often can't predict which before it runs. That variance makes cost-to-serve a distribution, not a number, and it's why naive per-task pricing is so dangerous. Cloud taught us to manage known variable costs; agents force us to manage uncertain ones.

Third, falling input prices don't automatically help you. In cloud, when compute got cheaper, your margins improved. In agents, cheaper tokens have repeatedly been eaten by agents simply doing more work per task, the counterintuitive dynamic behind why falling token prices didn't lower agent bills. A 50% drop in per-token cost means little if your agents respond by reasoning twice as hard.

So steal the cloud playbook, but don't worship it. The commit-and-burst logic, the utilization obsession, the per-customer attribution, the margin tiering, the FinOps muscle, all of that transfers cleanly and most GaaS vendors are leaving it on the table. The volatility and non-determinism, though, are yours to solve fresh. That's the part nobody can hand you a manual for.

Insights Most People Overlook

• Your model provider is running the AWS playbook on you. Provisioned throughput, committed-spend discounts, batch tiers, these are the same demand-certainty instruments AWS uses to lock in buyers, pointed at GaaS vendors. Most founders treat the model bill as a fixed input. It's actually a negotiable, structurable cost, and the vendors who negotiate it like a cloud commit will outprice the ones who don't.

• Idle capacity is the silent margin killer, and it gets worse as you grow up-market. Enterprise customers demand latency SLAs, which push you toward reserved throughput, which means you're now paying for capacity 24/7 against demand that's bursty by nature. The more "enterprise" you go, the more you look like a data center running at low utilization, and the more your gross margin depends on a metric your dashboard probably doesn't even show.

• Pass-through pricing isn't just low-margin, it's strategically suicidal. If your headline price is "model cost plus a markup," you've tied your revenue to a number your suppliers control and competitors can see. Every model price cut shrinks your absolute dollar margin even as your percentage holds. The cloud providers never sold raw compute as their value prop; they sold the layer above it. GaaS vendors pricing on token pass-through have learned nothing from that.

• Non-determinism means you should price on the cost distribution, not the average. If your cost-to-serve has a long tail, most tasks cheap, a few catastrophically expensive, averaging hides the customers who are bankrupting you. Cloud cost management assumes roughly knowable unit costs. Agent cost management requires you to price for the 95th percentile, or cap autonomy to clip the tail, which is precisely why some startups are quietly doing the latter.

• The FinOps function is a moat, not overhead. Early cloud-native winners built cost discipline as a competitive advantage, not a cost center, it let them undercut rivals while staying profitable. In GaaS, where COGS is the whole game, the vendor with mature cost attribution and commit optimization can price 20-30% below a sloppy competitor and still earn better margins. That's not a finance detail. That's the business.

References

• Google SRE Book, Site Reliability Engineering
• FinOps Foundation, The FinOps Framework
• McKinsey, Cloud cost optimization: A recipe to fund your digital transformation