Multi-agent governance: when your agents talk to each other
David Baum 13 min read The multi-agent inflection#
Single-agent governance is hard. Multi-agent governance is a different category of problem.
Gartner reported a 1,445% surge in multi-agent system inquiries from Q1 2024 to Q2 2025. Deloitte projects the autonomous AI agent market will reach $8.5 billion by 2026 and $35 billion by 2030, potentially $45 billion if enterprises get orchestration right. But IDC predicts that by 2026, 60% of AI failures will stem from governance gaps, not model performance.
The shift is structural. Single-agent governance handles the agent: what data it accesses, what actions it takes, what risk tier it belongs to. Multi-agent governance handles the interactions: what happens when Agent A delegates to Agent B, when Agent C trusts Agent D’s output, when five agents collaborate on a workflow that none of them individually controls.
Three interoperability protocols are now enabling this at enterprise scale: MCP (Anthropic) standardizes how agents access tools and resources, A2A (Linux Foundation, donated by Google) enables peer-to-peer agent collaboration and ACP (IBM) provides governance frameworks for enterprise multi-agent deployment. The infrastructure is ready. The governance isn’t.
Five risks that only exist in multi-agent systems#
Single-agent governance covers data sensitivity, decision authority and blast radius. Multi-agent systems introduce five additional risks that don’t exist when agents operate alone.
Risk 1: Orchestration drift#
The World Economic Forum’s December 2025 governance paper names this explicitly: “When agents are plugged into other agents without shared context or coordination logic, workflows can become brittle or unpredictable.”
Agent A’s output feeds Agent B, which feeds Agent C. But Agent A’s model gets updated, shifting its output format subtly. Agent B still parses it (mostly) but the edge cases pile up. Agent C receives degraded inputs and makes decisions based on data that looks right but isn’t. Nobody changed the workflow. The drift happened at the seams.
Orchestration drift is the multi-agent equivalent of configuration drift, but harder to detect because no single agent’s configuration changed. The drift lives in the interaction, not in any individual component.
Security asks: can someone break in? Governance asks: can we stand behind what this AI does, today and six months from now? As AI systems become more autonomous and pervasive in business operations, enterprises need more than just protective barriers. They need continuous oversight, accountability frameworks and the ability to demonstrate compliance with rapidly evolving regulations.
Risk 2: Semantic misalignment#
Two agents interpret the same instruction differently. The WEF paper warns this “can lead to conflicting actions or duplicated effort, with implications for safety, reliability and coordination.”
Agent A flags a support ticket as “resolved,” meaning the customer’s question was answered. Agent B reads “resolved” as “escalated to human review”; its internal schema uses the same word with a different meaning. The ticket closes without human review. Neither agent is wrong in isolation. Together they create a governance failure.
This risk scales with agent count. With 5 agents sharing a vocabulary, you have 10 potential misalignment pairs. With 20 agents, you have 190.
Risk 3: Cascading blast radius#
A single agent’s failure propagates through dependent agents, amplifying impact beyond the original agent’s scope. Kunal Singh’s OWASP blast radius analysis identified this pattern: “unchecked blast radius occurs when probabilistic agent behavior becomes persistent, trusted and shared across systems.”
When Agent A fails and Agent B trusts Agent A’s output, the blast radius is the union of both agents’ scopes. If Agent B feeds Agent C, the cascading blast radius encompasses all three. In a tightly coupled multi-agent workflow, a single agent’s misclassification or hallucination can ripple through the entire chain before any human sees it.
This is why risk classification must account for dependencies, not just individual agents. An agent that’s Tier 1 in isolation may be Tier 3 in practice because it feeds a chain that ends at a customer-facing Tier 4 agent.
Risk 4: Accountability diffusion#
When multiple agents contribute to a decision, who is accountable for the outcome? Palo Alto Networks’ governance guide identifies “accountability diffusion” as one of seven key agentic AI governance risks.
If Agent A gathers data, Agent B analyzes it, Agent C generates a recommendation and Agent D executes it and the execution causes harm, the accountability chain is unclear. Was the data wrong? Was the analysis flawed? Was the recommendation inappropriate? Was the execution out of scope? Each agent’s owner can point to the others and the audit trail becomes a maze of cross-references.
Multi-agent governance requires that the audit trail doesn’t just log what each agent did; it links the decision chain across all contributing agents so that the full causal path is reconstructable.
AI workflows are maturing and starting to cross corporate and cloud LLMs and agents. We are the only AI security vendor that can secure every AI interaction, everywhere, with a unified solution. The alternative is trying to stitch together secure workflows using network proxies, firewalls, DLP products and XDR agents. A complex mess.
Risk 5: Emergent behavior#
Multi-agent systems can exhibit behaviors that no individual agent was designed to produce. The MI9 framework identifies “emergent behaviors during execution” as the fundamental gap that runtime governance must address.
When agents negotiate, compete for resources or modify shared state, the outcomes can be unpredictable.
Emergent behavior is the hardest multi-agent risk to detect because it doesn’t appear in any individual agent’s behavioral baseline. It only manifests in the interaction patterns across agents.
The dependency graph: your multi-agent governance map#
The primary governance tool for multi-agent systems is the dependency graph: the map of which agents depend on which other agents, which data sources they share and which tools they invoke.
In your agent registry (such as Roval’s agent inventory) every agent records its dependencies: model providers, frameworks, data sources, integrations and, critically, other agents. These dependencies form a directed graph that enables four governance functions.
Impact analysis. Before changing Agent A (updating its model, modifying its configuration, changing its permissions) trace its downstream dependents in the graph. If Agent A feeds Agents B, C and D, the change assessment must include the impact on all four agents, not just Agent A.
Failure propagation. When Agent A fails, the graph shows which agents are affected immediately (direct dependents) and which are affected downstream (transitive dependents). The guardian agent can use this graph to proactively restrict or alert dependent agents before the failure cascades.
Compliance inheritance. If Agent A is classified as Tier 4 (Critical) and Agent B depends on Agent A’s output for consequential decisions, Agent B must be governed at Tier 3 minimum, even if Agent B’s own data sensitivity and decision authority would otherwise classify it as Tier 2. Risk flows downstream through dependencies. A risk classification system that accounts for these transitive dependencies is essential.
Audit trail linking. When reconstructing the decision chain for a multi-agent workflow, the dependency graph tells the auditor which agents’ logs to correlate. Rather than searching every agent’s audit trail, the graph provides the path: Agent A → Agent B → Agent C, with timestamps for each interaction.
An agent classified as Tier 1 (Low) in isolation may need Tier 3 governance in practice if it feeds a chain that ends at a critical system. This principle (that risk flows downstream through agent dependencies) is the most frequently missed governance requirement in multi-agent deployments. The dependency graph makes it visible. Without the graph, compliance inheritance is invisible and your lowest-risk agent may be the unmonitored link in your highest-risk chain.
Governing agent-to-agent communication#
The five risks above share a common root: ungoverned agent-to-agent communication. Five controls address this at the interaction boundary.
Message validation#
Every message between agents is validated against a schema before processing. Unexpected payload structures, missing fields, out-of-range values and type mismatches are rejected at the boundary, not silently processed by the receiving agent.
This prevents one agent’s malformed output from corrupting another agent’s state. It also catches orchestration drift early: if Agent A’s output format changes after a model update, the schema validation on Agent B’s input rejects the new format immediately rather than letting degraded data propagate through the workflow.
Authority delegation#
When Agent A delegates a task to Agent B, the delegation must be explicit and scoped. Agent B should not inherit Agent A’s full permissions; it should receive only the permissions necessary for the delegated task.
This is the least-privilege delegation principle. If Agent A is Tier 2 with access to internal data and it delegates a subtask to Agent B that involves restricted PII, the delegation is blocked. Agent A cannot grant permissions it has but Agent B’s classification doesn’t warrant. The governance layer validates every delegation against both agents’ risk classifications.
Cross-agent audit trails#
Every agent-to-agent interaction is logged with: source agent, target agent, message content hash, timestamp and the governance decision (permitted, blocked or escalated). This creates a linked audit trail across the full multi-agent workflow.
When an auditor asks “how did this decision get made?”, the cross-agent audit trail reconstructs the complete chain: Agent A received this input, produced this output, sent it to Agent B, which combined it with its own data, sent the result to Agent C, which executed the action. Every step is logged, every handoff is traced.
Interaction rate limits#
Agent-to-agent communication is rate-limited to prevent runaway loops. If Agent A and Agent B enter a feedback loop (each triggering the other repeatedly) the rate limiter breaks the loop before it consumes resources, triggers unintended actions or generates thousands of audit log entries that obscure real events.
Rate limits are configurable per agent pair and per direction. A tightly coupled pair (Agent A ↔ Agent B) might tolerate 100 interactions per minute. A loosely coupled pair (Agent A → Agent C) might be limited to 10. Any interaction that exceeds the limit is blocked and alerts the guardian agent.
Trust boundaries#
Not all agents trust each other equally. Agents within the same team or department may operate with higher mutual trust, passing data with minimal validation. Agents across organizational boundaries, across vendors or across security domains require stricter validation, additional logging and potentially human approval for inter-agent messages.
Trust levels are defined in the governance policy and enforced at the communication boundary. High-trust pairs skip detailed payload inspection (but still log). Medium-trust pairs get full schema validation. Low-trust pairs require human approval for each interaction above a configured threshold.
Three coordination patterns and their governance trade-offs#
Multi-agent systems follow one of three coordination patterns. Each has different governance implications, and maps to a different guardian agent deployment.
Centralized (supervisor)#
A single orchestrator agent coordinates all others: assigning tasks, routing data and aggregating results. The orchestrator is the single point of accountability.
Governance advantage: Clear accountability. Every decision flows through one agent, creating a single audit point. Policy enforcement can be concentrated at the orchestrator.
Governance risk: Single point of failure. If the orchestrator drifts, everything drifts. If it’s compromised, the entire workflow is compromised. The orchestrator must be classified at the highest risk tier of any agent it coordinates.
Guardian deployment: Sidecar guardian on the orchestrator. Every instruction the orchestrator issues is validated against policy before reaching the worker agents.
Decentralized (peer-to-peer)#
Agents communicate directly with each other without a central coordinator. Each agent decides independently which other agents to invoke and when.
Governance advantage: No single point of failure. The system is resilient to individual agent failures.
Governance risk: Accountability diffusion. No central audit trail. Harder to detect cascading failures because there’s no single place where the full workflow is visible. Emergent behavior is most likely in this pattern.
Guardian deployment: Mesh guardian. Every agent-to-agent interaction is intercepted and logged by a distributed guardian mesh that provides the cross-agent audit trail that the decentralized pattern otherwise lacks.
Hierarchical#
Manager agents coordinate sub-teams of worker agents. The manager delegates tasks, reviews outputs and reports to higher-level managers. This mirrors organizational structure.
Governance advantage: Balances accountability with resilience. Each manager is accountable for its sub-team. Failures are contained within sub-teams unless the manager propagates them.
Governance risk: Requires governance at every level of the hierarchy. Manager drift affects all workers in the sub-team. Communication between sub-teams (across managers) creates the same challenges as the decentralized pattern.
Start with the dependency graph. Map which agents depend on which other agents. Identify the chains that end at Tier 3-4 systems. Those are your highest-risk multi-agent workflows. Apply compliance inheritance: every agent in a high-risk chain must be governed at the tier warranted by the chain’s endpoint, not by its own classification in isolation. Then instrument the inter-agent boundaries with validation, logging, rate limits and trust controls.
Guardian deployment: Gateway guardian per sub-team. The gateway guardian monitors all communication within the sub-team and between the sub-team’s manager and the rest of the system.
Governance scales with interaction#
Every agent you add to your estate increases governance complexity, not linearly, but combinatorially. Two agents have one interaction to govern. Ten agents have 45 potential interaction pairs. Fifty agents have 1,225.
The dependency graph makes this complexity visible. The five inter-agent controls (message validation, authority delegation, cross-agent audit trails, rate limits, trust boundaries) make it governable. And the coordination pattern you choose (centralized, decentralized or hierarchical) determines where you place your guardian agents and how you enforce policy.
The agents will keep talking to each other. The question is whether you can see the conversation and govern it, before it creates a problem nobody anticipated.
Sources and further reading#
More in governance
The lethal trifecta: governing the three capabilities you can't remove
Five days in January 2026, four AI productivity tools shipped indirect prompt injection vulnerabilities. Same pattern in every case. Simon Willison named it in June 2025: private data access plus untrusted content plus external communication equals data exfiltration. You can't remove any leg without breaking the agent. Governance has to shift from prevention to containment.
David Baum The confused deputy problem: lessons from Meta's AI agent Sev-1
On March 18 2026, a Meta AI agent exposed restricted company and user data for two hours. The credentials were valid. The governance was not.
David Baum Guardian agents: when AI governs AI
Humans can't watch every agent, every action, every second. Guardian agents (dedicated policy engines that monitor, audit and enforce governance on other agents at runtime) resolve the tension between machine-speed execution and meaningful oversight.
David Baum