Procurement Red Flags When Buying AI Agents: The Warning Signs Every Buyer Should Catch Before Signing

Table of Contents

• Why Agent Procurement Breaks the Old Playbook
• Red Flag 1: The Vendor Can't Explain How the Agent Fails
• Red Flag 2: Pricing That Hides the Cost of Autonomy
• Red Flag 3: Evaluation Theater
• Red Flag 4: Vague Answers on Data, Identity, and Permissions
• Red Flag 5: No Human-in-the-Loop Story
• Red Flag 6: Contract Terms That Move All Risk to You
• Red Flag 7: Lock-In Disguised as Integration
• Red Flag 8: The Roadmap Is the Product
• A Practical Due-Diligence Checklist
• Insights Most People Overlook
• References

Why Agent Procurement Breaks the Old Playbook

For two decades, software procurement settled into a comfortable rhythm. You bought seats, you measured uptime, you negotiated a discount on the three-year deal, and the worst thing that usually happened was an outage or a clunky UI. The product did roughly the same thing on Tuesday that it did on Monday.

Agentic AI does not behave that way. When you buy an agent sold as a service, you are buying something that takes actions on your behalf, makes decisions across a workflow, and produces different outputs given the same inputs. The risk surface moves from "does it work" to "what does it do when it's wrong, and who's holding the bag when it is." That single shift is why a procurement process built for deterministic software waves through deals that should never close.

This is also why agent buying deserves its own discipline. We've written separately about the procurement process for buying agents as a structured workflow; this piece is the adversarial companion to it, the catalog of things that should make you pause. Think of the maturity question in parallel: organizations early on the enterprise agent-adoption maturity model tend to miss these flags entirely because they're still evaluating agents with a SaaS checklist. The flags below are ordered roughly from "most commonly missed" to "most expensive when ignored."

Red Flag 1: The Vendor Can't Explain How the Agent Fails

Ask any agent vendor a single question early: "Walk me through the last three ways this agent failed in production for a customer like us." A confident, specific answer is one of the strongest positive signals you can get. A deflection, "our agent has very high accuracy," "failures are extremely rare," "we haven't really seen that", is a serious red flag.

Every autonomous system fails. The vendors worth buying from know exactly how theirs does: it hallucinates a field in a specific document type, it loops when an API times out, it over-escalates during high volume, it silently drops the last step of a multi-step task. A mature vendor has a taxonomy of failure modes and a story for each one, detection, containment, fallback, and recovery. A vendor who can't produce that has either never run at scale or isn't watching closely enough to know. Both are reasons to slow down.

The deeper concern is observability. If the vendor can't tell you how the agent fails, ask whether you will be able to see it fail. Can you trace a single agent run end to end? Can you see which tools it called, what it retrieved, where it branched, and why it took the action it took? Agents that operate as black boxes turn every incident into a guessing game. This connects directly to the broader operational picture covered in why most agent pilots never reach production, opacity is one of the top reasons promising pilots die before scale.

Red Flag 2: Pricing That Hides the Cost of Autonomy

GaaS pricing is genuinely hard to reason about, and some vendors exploit that. The headline number, "$0.40 per task," "$2 per resolved ticket," "per successful outcome", is rarely the number you actually pay.

Watch for these specifically:

• Undefined units. What counts as a "task"? If an agent retries five times, is that one task or five? If it half-completes and a human finishes the job, who paid for what? Vendors who can't define the billable unit crisply are either disorganized or deliberately fuzzy.
• Outcome pricing with a self-graded exam. "Pay per successful outcome" sounds buyer-friendly until you ask who decides what "successful" means. If the vendor's own system adjudicates success, you have a fox-guarding-henhouse problem.
• Token and inference pass-through. Some agents make many model calls per task. If underlying model costs pass through to you uncapped, a single complex workflow can cost 10x a simple one with no warning.
• Minimums and overage cliffs. Generous unit pricing paired with a high monthly minimum means you're effectively on a subscription wearing a usage-pricing costume.

Andreessen Horowitz has written usefully on how the economics of AI agents differ from traditional SaaS, and the takeaway for buyers is simple: model your real workload, not the demo. Run the vendor's pricing against your actual task mix, including the messy long-tail cases, before you believe any per-unit figure. This ties into the broader discipline of measuring agent ROI that a CFO will believe, a price you can't predict is a price your finance team won't trust.

Red Flag 3: Evaluation Theater

Every agent vendor will show you a benchmark. The red flag is when the benchmark conveniently can't be reproduced on your data.

Evaluation theater takes a few forms. There's the cherry-picked demo, where the agent handles three perfectly clean cases and you never see a messy one. There's the proprietary benchmark with no published methodology, impressive numbers, no way to verify them. And there's the refusal to run a structured pilot on your representative data with your definition of correct.

The fix is to insist on a real evaluation you control. Bring your own test set, including the ambiguous, adversarial, and edge cases that actually break things. Define what "correct" means before the agent runs, not after. Measure not just accuracy but failure cost: a 95% success rate sounds great until you learn the 5% are your highest-value transactions. Reputable guidance on AI risk, including the NIST AI Risk Management Framework, pushes buyers toward exactly this kind of context-specific, measurable evaluation rather than vendor-supplied scorecards.

A vendor who welcomes a rigorous, buyer-controlled pilot is showing confidence. A vendor who resists one, stalls on it, or insists their internal numbers should be enough is telling you something. Believe them.

Red Flag 4: Vague Answers on Data, Identity, and Permissions

Agents act inside your systems. That means they need credentials, access, and the ability to take actions, and that means the security questions are categorically more serious than for passive software.

Push hard on three things:

Data handling. Where does your data go? Is it used to train the vendor's models or shared models? Can you opt out? What's retained, for how long, and where? "We take security seriously" is not an answer; a data-flow diagram is.

Identity and scope. How does the agent authenticate into your systems, and what's the blast radius if its credentials leak? An agent with a broad standing service account is a liability waiting to happen. The strong pattern is scoped, least-privilege, auditable access, ideally per-action authorization rather than a master key. Vendors who shrug at this question haven't thought about agent security seriously, which is its own answer.

Action boundaries. What can the agent actually do, and what's hard-walled off? Can it move money, delete records, send external communications, or change configurations? The absence of clearly enforced action limits is one of the more dangerous red flags on this list, because the failure mode isn't a wrong answer, it's a wrong action you can't undo.

If these questions get hand-waved, that's frequently a sign you're looking at a vendor whose agent will end up in the shadow agents and the governance scramble category once it's deployed, operating with more access than anyone intended and no one quite tracking it.

Red Flag 5: No Human-in-the-Loop Story

Full autonomy makes for a great pitch and a risky deployment. Be wary of any vendor whose answer to "when does a human step in?" is essentially "they don't have to."

The mature position is graduated autonomy: the agent handles what it's reliably good at, escalates what it isn't, and the threshold is configurable by you. A good vendor will talk naturally about confidence thresholds, escalation paths, approval gates for high-stakes actions, and how the human oversight workload changes as trust builds. They'll have thought about the human-oversight staffing model because their existing customers had to.

A vendor with no human-in-the-loop story is selling you either overconfidence or a future incident. The point of oversight isn't to distrust the agent forever; it's to earn trust deliberately, in line with how teams actually move along the trust-building curve when they start delegating real work to agents. Ask specifically: how do I dial autonomy up over time, and what evidence does the system give me to justify doing so?

Red Flag 6: Contract Terms That Move All Risk to You

Read the contract like the agent is going to make an expensive mistake, because eventually it will. The red flags here are quieter than the technical ones but often more costly.

Look for:

• Liability caps set at the price of a coffee. If the vendor's maximum liability is "fees paid in the last 30 days," they are not standing behind the agent's actions in any meaningful way.
• No indemnification for agent-caused harm. If the agent takes a wrong action that damages a customer relationship or violates a regulation, who's responsible? Silence in the contract means you are.
• No SLAs that fit agents. Traditional uptime SLAs are necessary but not sufficient. You want commitments on accuracy floors, escalation responsiveness, and incident communication, not just "the API was reachable."
• Unilateral model-change clauses. Many agents sit on top of third-party foundation models. If the vendor can swap the underlying model without notice, your agent's behavior can change overnight with no recourse. Insist on change notification and a regression-testing window.

The Harvard Business Review's ongoing coverage of how companies are governing generative AI reinforces a theme worth internalizing: governance and accountability have to be contractual, not aspirational. A handshake about "partnership" is not a substitute for terms that allocate risk honestly.

Red Flag 7: Lock-In Disguised as Integration

Deep integration is genuinely valuable, an agent wired into your systems does more useful work. But there's a fine line between integration and capture, and some vendors blur it on purpose.

The question to ask is: what happens when we want to leave? If your workflows, prompts, evaluation history, and accumulated agent "memory" all live in a proprietary format you can't export, switching costs become a moat the vendor built around you, not a feature you bought. Watch for refusal to provide data export, no clarity on who owns the fine-tuned or customized agent configuration, and architectures that make it impossible to run the same workflow against a different model or vendor.

This matters more as you scale, because a single locked-in agent is an annoyance but a fleet of them is a strategic problem, exactly the dynamic that drives vendor management when you run 30 different agents into a major operational headache. Negotiate portability and ownership up front, while you still have leverage. After deployment, the leverage is gone.

Red Flag 8: The Roadmap Is the Product

A lot of agent startups are selling ambition. The demo works on the happy path, the founder is compelling, and the real capability is "coming in Q3." Sometimes that's a reasonable bet on a fast-moving company. Often it's a red flag.

The tell is when you can't get a straight answer about what works today, in production, for customers your size, at your volume. Ask for references you can actually call, not logos on a slide, but operators running the agent in conditions like yours. Ask what breaks at scale. Ask what the agent specifically cannot do yet, and listen for whether the vendor can name limitations crisply (good sign) or gets defensive (bad sign).

McKinsey's research on the state of AI and where value is actually being captured repeatedly lands on the same point: the gap between AI pilots and production value is wide, and a lot of it comes down to operational readiness the demo never reveals. Buy the product that exists, not the one in the pitch deck.

A Practical Due-Diligence Checklist

If you take nothing else, take these questions into your next agent vendor conversation:

1. Walk me through the last three production failures for a customer like us.
2. Show me a full trace of one agent run, every tool call and decision.
3. Define your billable unit precisely. Now price my real workload, edge cases included.
4. Who decides what counts as a "successful outcome," and can I audit it?
5. Let me run a pilot on my data with my definition of correct.
6. Diagram where my data goes and confirm whether it trains your models.
7. How does the agent authenticate, and what's the blast radius if it's compromised?
8. What actions are hard-walled off? Can it move money or delete records?
9. When does a human step in, and how do I change that threshold?
10. What's your liability cap and indemnification for agent-caused harm?
11. Can I export everything and leave? Who owns the customized configuration?
12. What works in production today versus what's on the roadmap?

A vendor who answers all twelve well is rare and worth a premium. A vendor who flinches at four or five is telling you where the deal will go wrong.

Insights Most People Overlook

The most dangerous red flag is a vendor who's too polished on autonomy. Buyers instinctively distrust vendors who hedge. But in agent procurement, a vendor who confidently promises near-total autonomy with minimal oversight is usually more dangerous than one who's candid about limitations. Smooth confidence about a probabilistic system that takes irreversible actions is a sales posture, not an engineering reality. Reward the vendor who tells you what their agent can't do.

Outcome-based pricing can be a red flag, not a green one. The industry treats per-outcome pricing as the buyer-friendly gold standard, and sometimes it is. But it quietly incentivizes the vendor to define "outcome" narrowly and to optimize the agent toward billable events rather than your actual goals. If success is self-graded and the agent is tuned to maximize "resolutions," you can end up paying more for an agent that closes tickets fast rather than well. Outcome pricing is only as good as your ability to audit the outcome.

The cheapest agent often has the highest total cost of ownership. A low per-task price frequently signals a thin vendor, little observability, no human-oversight tooling, weak integration support. You'll rebuild all of that internally, and the hidden labor of babysitting an opaque agent dwarfs the licensing savings. This is the gap between sticker price and total cost of ownership for an enterprise agent program, and it's where most first-time agent buyers get the math wrong.

Reference customers who only run pilots are a yellow flag. When a vendor's references are all in pilot or "expanding" phase, dig in. It may mean the agent demos beautifully and then stalls before production, the pilot-purgatory pattern. Ask references one question: "Is this in full production, and would you have bought it again?" The pause before the answer tells you more than the answer.

Watch who owns the agent's memory. As agents accumulate context, corrections, and workflow history, that accumulated "experience" becomes genuinely valuable, and genuinely sticky. Vendors rarely volunteer who owns it. If the agent gets materially better the longer it runs, and that improvement is locked in the vendor's system, you're not just renting capability, you're funding an asset you don't own. Settle ownership before deployment, never after.

References

• The Economics of Generative AI, Andreessen Horowitz
• AI Risk Management Framework, NIST
• Managing the Risks of Generative AI, Harvard Business Review
• The State of AI, McKinsey & Company