Agent Governance Frameworks: The Standards Taking Shape (and the Ones That Won't)

Table of Contents

• Why Agent Governance Is a Different Problem
• The Frameworks Actually in Play
• NIST AI RMF and the Agentic Stretch
• The EU AI Act as a De Facto Standard
• ISO/IEC 42001 and the Management-System Angle
• Vendor and Consortium Efforts
• The Five Control Domains Every Framework Converges On
• Building a Governance Framework That Survives Contact
• Where GaaS Buyers and Vendors Diverge
• Insights Most People Overlook
• References

Why Agent Governance Is a Different Problem

Most of the governance machinery enterprises already own was built for two things: software that does exactly what it's told, and people who can be held responsible when they don't. An autonomous agent is neither. It improvises within a goal, it calls tools and other agents, and it makes thousands of small decisions no human reviewed in advance. That breaks the assumptions baked into traditional IT controls.

Consider a concrete example. A change-management board approves a deployment because it can read the code and predict the behavior. Now hand that board an agent that books vendor payments based on natural-language instructions, pulls live data from three SaaS tools, and decides on its own which invoices to prioritize. The board cannot pre-approve every decision path because the path is generated at runtime. Governance has to shift from reviewing actions to constraining the space of possible actions and reconstructing what happened afterward. That single shift, from prevention-by-review to constraint-plus-forensics, is the conceptual core of every serious agent governance framework, and it's why bolting agents onto an existing GRC program rarely works cleanly.

It's also why this topic sits at the center of the Trust, Safety, Security and Governance beat in the broader GaaS conversation. Governance is the connective tissue between the narrower problems, identity for non-human actors, scoped permissions, audit logs, kill switches, and it determines which of those controls you're actually required to implement.

The Frameworks Actually in Play

There is no FDA-style approval body for agents, and there won't be one soon. What exists is a set of overlapping frameworks, each designed for a slightly different purpose, that buyers and regulators are stitching together.

NIST AI RMF and the Agentic Stretch

The NIST AI Risk Management Framework is the most cited starting point in U.S. enterprises, largely because it's voluntary, vendor-neutral, and maps cleanly onto how risk teams already think. Its four functions, Govern, Map, Measure, Manage, translate reasonably well to agents. But the RMF was written for models, not for systems that act. The "Measure" function assumes you can characterize a model's behavior through testing; an agent's behavior is emergent and depends on the tools it's wired to, so a measurement done in a sandbox tells you little about production.

The honest read: NIST AI RMF gives you vocabulary and a process skeleton, not agent-specific controls. The companion Generative AI Profile that NIST published closes some of the gap by naming risks like confabulation and information-security exposure, but anyone using the RMF for agents is still doing original work to define what "manage" means when the system can take irreversible actions.

The EU AI Act as a De Facto Standard

For anyone selling into Europe, or selling to multinationals, the EU AI Act is the framework that actually has teeth. It classifies AI systems by risk tier, and a surprising number of agentic use cases land in the "high-risk" bucket: anything touching credit decisions, hiring, essential services, or critical infrastructure. High-risk systems carry hard obligations, risk management systems, logging, human oversight, accuracy and robustness documentation, and the logging requirement in particular is reshaping how GaaS vendors build, because it makes the audit trail a product feature rather than an afterthought (a theme explored in #168).

The Act's general-purpose AI provisions add another layer for the foundation-model providers underneath the agents. The net effect is that the EU AI Act functions as a de facto global standard the same way GDPR did: it's easier to build one compliant product than to maintain two. Vendors who treated it as a European-only problem in 2024 are now retrofitting, and retrofitting governance is expensive.

ISO/IEC 42001 and the Management-System Angle

ISO/IEC 42001, the AI management system standard, is the quietest of the three but arguably the most consequential for procurement. It's certifiable. That matters because enterprise buyers increasingly want a stamp they can put in a vendor file, the same way SOC 2 became table stakes for SaaS. 42001 doesn't tell you how to govern an agent; it tells you that you need a documented, audited management system that does the governing, policies, roles, continual improvement. For GaaS vendors, getting 42001-certified is becoming a sales-enablement move as much as a safety one, which ties directly into the certifications-buyers-require discussion.

Vendor and Consortium Efforts

Below the formal standards sit the fast-moving, less-settled efforts: cloud providers' responsible-AI frameworks, the Cloud Security Alliance's agent-focused guidance, the emerging work around agent identity standards, and protocol-level security models like the one debated around MCP (#187). These move at software speed and fill the gaps the slow standards bodies leave open. The risk is fragmentation, five vendor frameworks that don't interoperate, which is exactly the failure mode that motivated proposals for a shared trust certification for agent vendors.

The Five Control Domains Every Framework Converges On

Strip away the branding and the serious frameworks agree on roughly five domains. If a "governance framework" doesn't address all five, it's incomplete.

Identity and authority. Every agent action should trace to a verifiable agent identity and a human or system that delegated the authority. This is harder than it sounds, you're authenticating a non-human actor that may spawn sub-agents, and it's the foundation everything else rests on. The confused-deputy problem lives here: an agent acting with borrowed authority it shouldn't have.

Permission scoping. Least privilege, applied to agents, means an agent gets the narrowest set of tools and data access its task requires, and nothing more. Frameworks increasingly demand runtime scoping, not just provisioning-time, because an agent's needs shift mid-task. This is the practical heart of scoped-permission design.

Oversight and intervention. Human-in-the-loop, human-on-the-loop, or human-over-the-loop, the framework has to specify which, for which actions, and provide a real intervention path. A kill switch that can actually stop a running fleet is non-negotiable for high-risk deployments, and "we'll just turn off the API key" is not a credible answer when an agent has in-flight transactions.

Auditability and forensics. Because you can't pre-review every action, you must be able to reconstruct any of them. This means decision-level logging, inputs, tool calls, reasoning traces where available, outputs, retained in a tamper-evident form. Regulators will ask for it; the EU AI Act already does.

Accountability assignment. Someone owns the agent. The emerging consensus, and increasingly a requirement, is a named human accountable owner for every deployed agent, a person who answers for what it does. This is the control that keeps "no single human controls it" from becoming "no single human is responsible for it."

Notice that four of the five are about reconstruction and constraint, not about making the agent smarter or more reliable. Governance is largely orthogonal to capability. A more capable agent under weak governance is more dangerous, not less.

Building a Governance Framework That Survives Contact

The frameworks that survive an actual incident share a few traits, and they're not the traits that show up in glossy maturity models.

First, they're evidence-first. The test of any control isn't whether you have a policy; it's whether you can produce, on demand, proof that the control operated. If your kill-switch policy can't be backed by a log showing it was tested last quarter, it's a slide, not a control. Build every control with its evidence artifact defined up front.

Second, they treat irreversibility as the key risk axis. Not all agent actions are equal. Sending a draft email for human review is cheap to get wrong; wiring money or deleting a production database is not. Effective frameworks tier their controls by reversibility and blast radius, concentrating human oversight on the actions you can't take back. A flat "all agent actions need approval" policy gets ignored within a week because it's unworkable; a tiered one survives.

Third, they assume the agent will be attacked. Prompt injection, jailbreaking, and tool poisoning aren't edge cases, they're the threat model. A governance framework that only considers the agent failing honestly, and never the agent being manipulated into failing, is missing half the surface (#166, #189 cover this terrain). McKinsey and others have noted that organizations capturing real value from AI are disproportionately the ones that built risk and governance discipline alongside deployment rather than after it.

Fourth, they have a governance owner with actual authority. A governance committee that can't stop a deployment is theater. The organizations doing this well are standing up agent governance committees with veto power and a clear charter, and pairing them with the named-owner requirement so accountability doesn't evaporate into "the committee."

Where GaaS Buyers and Vendors Diverge

Here's the friction that doesn't get said out loud. Buyers and vendors want different things from governance, and the standards are caught in the middle.

Buyers want governance to be the vendor's problem, certifications, attestations, contractual liability, so they can deploy quickly and point at the vendor if something breaks. Vendors want governance to be a shared-responsibility model, like cloud security, where the buyer is responsible for how they configure and scope the agent, and the vendor is only responsible for the platform. The contracts being signed in 2026 are mostly resolving this in the vendor's favor with liability waivers that shift configuration risk to the buyer, which means the buyer's internal governance matters more than the vendor's marketing implies.

This is also why SOC 2 and ISO 42001 certifications are becoming the negotiation currency. A buyer can't audit a vendor's agents directly, so they proxy trust through certifications and security questionnaires. The vendors winning enterprise deals are the ones treating compliance as a product capability, a positioning play sometimes called compliance-as-a-feature, rather than a cost center. The governance framework, in other words, isn't just a safety artifact. It's increasingly the thing that closes the deal.

Insights Most People Overlook

The "framework" most companies have is a screenshot of a maturity model with no evidence behind it. The gap between having a governance framework and being able to operate one in an audit is enormous. The single highest-leverage move isn't adopting a fancier framework, it's defining, for each control you claim, the specific log or artifact that proves it ran. Most governance programs fail this test on day one.

Governance maturity and agent capability are nearly independent, and that's a trap. Teams assume a more sophisticated agent needs less governance because it "makes fewer mistakes." The opposite is true: a more capable agent takes more consequential actions faster, so weak governance scales the downside. The most dangerous deployments are high-capability agents inside organizations that mistook capability for safety.

The real standards war is about the audit log format, not the policy language. Everyone agrees agents need oversight and accountability, that's cheap consensus. The genuinely contested ground is what an audit trail must contain to be admissible to a regulator or usable in a forensic investigation. Whoever's log schema becomes the default will shape the entire ecosystem, because tooling, insurance underwriting, and incident response will all standardize around it. Watch that space more than the policy debates.

Reversibility is a better organizing principle than risk tier. The high-risk/low-risk classification from the EU AI Act is useful for compliance but clumsy for engineering. A control framework organized around "can this action be undone, and how big is the blast radius?" produces more defensible decisions about where to put humans in the loop than a static system-level risk label ever will.

The accountability gap is being papered over with named owners who lack real control. Requiring a human accountable owner is the right instinct, but in practice that person often can't actually predict or constrain what the agent does. We're creating roles that carry responsibility without commensurate control, a setup that's unstable and will get tested hard the first time an agent causes real harm and the named owner says, truthfully, "I had no way to know it would do that."

References

• NIST AI Risk Management Framework, National Institute of Standards and Technology
• EU AI Act: Explorer and Full Text, The Artificial Intelligence Act (EU 2024/1689)
• ISO/IEC 42001, Artificial Intelligence Management System, International Organization for Standardization
• The State of AI, McKinsey & Company, QuantumBlack