AI agent governance glossary

Defined terms covering agent registries, risk classification, compliance certification, drift detection and LLM monitoring.

97 terms

A

Agent card

A structured metadata record for a single AI agent — capturing its name, owner, risk tier, models used, data access, deployment status, and compliance certifications. The digital twin of an agent in your registry.

Agent drift

Behavioral change in an AI agent that occurs without any code modification — caused by upstream model updates, tool API changes, data quality shifts, modifications to dependent agents, or evolving business context. Distinct from model drift, which tracks statistical distribution shift in input data.

Agent identity

The unique, verifiable identity assigned to an AI agent for authentication, authorization, and audit trail purposes. Distinct from non-human identity in that it encompasses the agent's behavioral fingerprint, capability boundaries, and delegation authority — not just its credentials.

Agent lifecycle

The stages an AI agent moves through from initial registration and risk classification, through compliance certification and deployment, to ongoing monitoring, recertification, and eventual decommissioning.

Agent registry

A centralized inventory of every AI agent operating in an organization — who owns it, what it does, which models it calls, what data it touches, and its current compliance status. The foundation of agent governance.

Agent sprawl

Uncontrolled proliferation of AI agents across teams and environments, without central visibility or governance. Often the result of rapid adoption outpacing security and compliance processes.

Agentic AI

AI systems that can autonomously plan, reason, use tools, and take actions to accomplish goals — as opposed to models that only generate predictions or text in response to prompts. The category that agent governance exists to govern.

AgentOps

The emerging operational discipline for managing autonomous AI agents throughout their lifecycle. Extends MLOps to cover agent-specific requirements — behavioral observability, tool access governance, human-in-the-loop escalation, compliance certification, and decommissioning.

AI Act (EU)

The European Union's regulation on artificial intelligence (Regulation 2024/1689), establishing risk-based requirements for AI systems including conformity assessments, risk management, and post-market monitoring.

AI governance

The policies, processes, and controls an organization uses to manage AI systems responsibly — covering risk assessment, compliance, monitoring, accountability, and transparency across the AI lifecycle.

AI literacy

The knowledge and skills required to understand, interact with, and oversee AI systems. Article 4 of the EU AI Act mandates that providers and deployers of AI systems ensure sufficient AI literacy among their staff, proportionate to the risk level of the systems they operate.

Anomaly detection

Automated identification of unusual patterns in agent behavior, such as unexpected request volumes, atypical model usage, or sudden cost spikes that may indicate misconfiguration, abuse, or compromise.

Audit trail

A chronological, tamper-evident record of all actions taken on or by an AI agent — configuration changes, compliance decisions, policy violations, and remediation steps. Required by most compliance frameworks.

Auto-expiry

A governance mechanism that automatically invalidates an agent's compliance certification after a time-bound period — typically set by risk tier (e.g., 90 days for critical, 180 for high, 365 for low). Forces periodic recertification.

Automation bias

The tendency for human operators to over-rely on AI agent outputs and rubber-stamp automated decisions without meaningful review. A governance risk explicitly addressed in EU AI Act Article 14, which requires human oversight measures designed to counteract this bias.

B

Behavioral baseline

A statistical profile of an agent's normal operating patterns — tool call frequency, API usage, data access patterns, response times — established over time and used to detect anomalies that may indicate drift, misconfiguration, or compromise.

Blast radius

The potential scope of damage if an AI agent fails, is compromised, or behaves unexpectedly. Determined by the agent's data access, autonomy level, integration points, and the sensitivity of downstream systems.

C

Cedar

An open-source policy language developed by Amazon Web Services for defining and enforcing authorization policies. Used alongside OPA/Rego in policy-as-code approaches to AI agent governance, with a focus on attribute-based access control and verifiable policy evaluation.

Circuit breaker

An automated safety mechanism that halts an AI agent's operations when policy violations or anomalies exceed a defined threshold. Blocks all further tool calls until an administrator reviews and resets. The runtime kill switch required by EU AI Act Article 14.

CMDB for AI

Applying configuration management database principles to AI agents — treating each agent as a managed asset with tracked attributes, dependencies, change history, and compliance status.

Compliance certification

The process of formally verifying that an AI agent meets the requirements of one or more regulatory frameworks, resulting in a time-bound certification that must be renewed periodically.

Compliance drift

Gradual divergence of an AI agent's actual configuration, behavior, or operational context from its certified compliance baseline — often caused by model updates, prompt changes, or scope creep.

Conformity assessment

A systematic evaluation, required under the EU AI Act and other frameworks, to determine whether a high-risk AI system meets applicable regulatory requirements before it can be deployed.

Continuous compliance

A governance approach where an agent's compliance status is evaluated on an ongoing basis rather than at a single point in time. Combines drift detection, automated recertification triggers, and real-time policy enforcement to maintain compliance between formal audit cycles.

Coordination breakdown

A failure mode in multi-agent systems where agents cannot properly synchronize actions, share state, or hand off tasks — resulting in workflow failures and inconsistent outcomes. Accounts for roughly 37% of multi-agent production failures.

Cost attribution

Mapping LLM API spending back to specific agents, teams, projects, or business units. Enables chargeback models and identifies which agents drive disproportionate cost.

D

Data classification

Categorizing data by sensitivity level (public, internal, confidential, restricted) to determine what information an AI agent should be permitted to access, process, or transmit.

Data exfiltration

Unauthorized extraction of sensitive data through an AI agent — whether through prompt content, model responses, or side channels. A primary concern in LLM monitoring and policy enforcement.

Decision chain

The sequence of reasoning steps, tool invocations, and intermediate decisions an AI agent takes to complete a task. Tracing decision chains is essential for debugging failures, attributing accountability, and meeting audit requirements.

Decommissioning

The controlled process of retiring an AI agent — revoking access, archiving audit logs, notifying stakeholders, and updating the registry. The final stage of the agent lifecycle.

Dependency graph

A visual and queryable map of how AI agents connect to each other, to APIs, databases, models, and downstream systems. Reveals blast radius, cascading failure paths, and single points of failure across the agent estate.

Digital worker

An autonomous AI agent that functions as a digital employee — with its own identity, credentials, and access to the same tools and systems that humans use. A framing that emphasizes the need for identity governance, access control, and accountability at parity with human workers.

Drift detection

Continuous monitoring that identifies when an agent's configuration, behavior, or compliance posture deviates from its certified baseline. Triggers alerts and may require recertification.

E

EHDS

The European Health Data Space: an EU regulation establishing rules for the use, access, and sharing of health data across member states. Introduces specific requirements for AI systems processing health data, including purpose limitation, data quality standards, and cross-border interoperability obligations.

Escalation framework

A tiered system defining when an AI agent must notify, consult, or hand off to a human — based on risk level, confidence thresholds, or policy rules. Ranges from fully autonomous operation to mandatory human approval.

EU AI Act risk tiers

The four risk categories defined by the EU AI Act: unacceptable (banned), high-risk (requires conformity assessment), limited risk (transparency obligations), and minimal risk (no specific requirements).

F

Fail-open

A design pattern where a monitoring proxy or policy enforcement point allows requests to pass through if it encounters an error, preserving application availability at the cost of temporary unmonitored traffic.

Foundation model

A large AI model (such as GPT-4, Claude, or Llama) pre-trained on broad data and adapted for downstream tasks. Most enterprise AI agents call foundation models via API rather than running their own trained models.

Framework mapping

The process of linking an agent's properties and controls to specific requirements across multiple compliance frameworks (e.g., EU AI Act, ISO 42001, SOC 2), identifying gaps and overlaps.

G

Goal hijacking

An attack or failure mode where an AI agent's objectives are redirected toward unintended outcomes — through prompt injection, adversarial input, or environmental manipulation. The agent pursues a goal its operators never intended.

Guardian agent

An AI agent whose purpose is to monitor, audit, or constrain other agents — enforcing policy compliance, detecting anomalous behavior, and triggering escalation or circuit breakers. A governance-layer agent that watches the agent estate.

Guardrail

A runtime constraint applied to an AI agent's behavior — such as blocking certain prompt patterns, enforcing output filters, or limiting model access based on data classification.

H

Hallucinated topology

The failure mode where an AI agent reasons about infrastructure it has never been given a verified map of — assuming API endpoints are stable, data sources available, and permissions unchanged. The agent's mental model of its environment diverges from reality.

Hallucination

When an AI model generates confident but factually incorrect, fabricated, or nonsensical output. A governance concern because agents may act on hallucinated information — executing decisions based on facts that don't exist.

High-risk AI system

An AI system classified under the EU AI Act as posing significant risk to health, safety, or fundamental rights. Subject to mandatory conformity assessment, risk management, human oversight, and post-market monitoring.

Human-in-the-loop

A governance requirement that certain AI agent actions — particularly high-risk decisions — must be reviewed and approved by a human before execution. Often mandated by compliance frameworks.

I

Incident response

The structured process for detecting, containing, investigating, and remediating AI agent failures, policy violations, or security breaches. Includes root cause analysis and post-incident review.

ISO 42001

The international standard for AI management systems, published by ISO/IEC. Specifies requirements for establishing, implementing, maintaining, and improving an AI management system within an organization.

K

Kill switch

A mechanism to immediately stop an AI agent's operations. Required under EU AI Act Article 14 for high-risk systems. Implementations range from manual admin controls to automated circuit breakers triggered by policy violation thresholds.

L

Least privilege

The security principle that an AI agent should have only the minimum permissions necessary to perform its intended function — no broader data access, no extra tool permissions, no unnecessary API scopes. Applied at registration and enforced continuously.

LLM monitoring

Real-time observation of all traffic between AI agents and language model APIs — capturing prompts, responses, token counts, latency, cost, and policy compliance for every request.

LLM proxy

A lightweight intermediary deployed between AI agents and language model APIs that captures every request — logging tokens, latency, cost, and content for audit, policy enforcement, and cost attribution.

LLMOps

The operational discipline for managing large language model deployments — covering prompt management, model versioning, cost optimization, latency monitoring, and output quality evaluation. A precursor to AgentOps, focused on the model layer rather than the autonomous agent layer.

M

Model drift

Statistical shift in an AI model's behavior over time, caused by changes in the underlying model weights (provider updates), input data distributions, or prompt effectiveness. Distinct from agent drift, which encompasses broader behavioral changes including tool usage patterns and decision-making.

Model inventory

A registry of all language models and AI models in use across an organization — including provider, version, capabilities, cost per token, and which agents are authorized to call each model.

Multi-agent system

An architecture where multiple AI agents collaborate, delegate tasks, or share information to accomplish complex goals. Introduces governance challenges around inter-agent communication, shared state, cascading failures, and distributed accountability.

N

NIS2 Directive

The EU Network and Information Security Directive (2022/2555), which strengthens cybersecurity requirements for essential and important entities across critical sectors. Applies to AI agents operating within critical infrastructure, requiring incident reporting, risk management, and supply chain security measures.

NIST AI RMF

The NIST Artificial Intelligence Risk Management Framework — a voluntary US framework providing guidance on governing, mapping, measuring, and managing AI risks throughout the AI lifecycle.

Non-human identity

A digital identity assigned to an AI agent (rather than a human user) for authentication, authorization, and audit purposes. Enables least-privilege access control, ownership tracking, and accountability tracing across the agent lifecycle.

Notified body

An organization designated by an EU member state to assess whether high-risk AI systems meet the requirements of the EU AI Act before they can be placed on the market. Conducts conformity assessments, reviews technical documentation, and issues certificates of conformity.

O

Observability

The ability to understand the internal state and behavior of AI agents through external signals — logs, metrics, traces, and alerts. Goes beyond monitoring to enable debugging and root cause analysis.

OPA

Open Policy Agent: an open-source, general-purpose policy engine that uses the Rego language to define and enforce policies as code. Widely adopted in AI agent governance for runtime authorization decisions, data access controls, and compliance rule evaluation across heterogeneous agent estates.

Orchestration

The coordination, sequencing, and management of multiple agents, tools, and workflows to accomplish complex tasks. Includes dependency management, error handling, and task handoffs. Multi-agent systems without proper orchestration experience failure rates of 41% to 87%.

Orphaned agent

An AI agent that continues operating after its human owner or sponsor has left the organization, changed roles, or abandoned the project. Retains its original access privileges and operates without oversight — a common vector for security incidents.

Owner

The person or team accountable for an AI agent's behavior, compliance, and lifecycle management. Typically assigned during registration and displayed on the agent card in the registry.

P

PII detection

Automated scanning of prompts and responses for personally identifiable information — email addresses, phone numbers, social security numbers, credit card numbers — before they reach a language model API.

Policy violation

An event where an AI agent's behavior breaches a defined governance rule — such as sending PII to an external model, exceeding a cost threshold, or calling an unauthorized API.

Policy-as-code

Governance rules written as executable code that can be versioned, tested, and enforced automatically against agent behavior. Enables consistent, auditable policy enforcement at scale.

Post-market monitoring

Ongoing surveillance of an AI system after deployment, as required by the EU AI Act. Includes tracking incidents, performance degradation, emerging risks, and compliance drift.

Production gate

A governance checkpoint that an AI agent must pass before advancing to the next lifecycle stage — particularly before reaching production. Typically requires active compliance certification, owner assignment, and risk classification above a threshold tier.

Prompt injection

An attack where malicious instructions are embedded in input data to manipulate an AI agent's behavior — potentially bypassing safety filters, extracting training data, or performing unauthorized actions.

R

RACI model

A governance framework (Responsible, Accountable, Consulted, Informed) applied to AI agent operations. Defines who is responsible for each agent's decisions, who is accountable for outcomes, who should be consulted, and who should be informed. Enterprises with clearly defined RACI models for AI agents resolve incidents 54% faster.

RAG

Retrieval-Augmented Generation: an architecture pattern where an AI agent retrieves relevant documents from a knowledge base before generating a response. Governance concerns include data access controls on the retrieval layer and accuracy of retrieved context.

Recertification

The process of re-evaluating an AI agent's compliance status after a material change — such as a model update, scope expansion, or detected drift — to maintain valid certification.

Red teaming

Adversarial testing of AI agents by simulating attacks, edge cases, and misuse scenarios to identify vulnerabilities before deployment. Covers prompt injection, tool misuse, privilege escalation, data exfiltration, and behavioral boundary violations.

Rego

The declarative query language used by Open Policy Agent (OPA) to define governance policies as code. Policies written in Rego can evaluate agent actions against compliance rules, data access controls, and operational constraints at runtime — enabling automated, auditable policy enforcement.

Risk classification

The process of assigning a risk tier to an AI agent based on factors like data sensitivity, decision autonomy, blast radius, regulatory exposure, and the criticality of downstream systems.

Risk tier

A classification level (e.g., low, medium, high, critical) assigned to an agent based on data access, autonomy, blast radius, and regulatory exposure. Determines the required level of governance controls.

Runtime enforcement

The active blocking, modification, or escalation of agent actions in real time based on governance policies — as opposed to passive monitoring that only logs violations after the fact. Includes request blocking, PII redaction, model switching, and automatic escalation to human oversight.

Runtime policy

A governance rule enforced in real time as an AI agent operates — as opposed to design-time or deployment-time checks. Examples include PII blocking, cost limits, and model allow-lists.

S

SaaS-embedded agent

An AI agent deployed within a third-party SaaS platform that an organization uses but does not directly control. The hardest agent type to govern due to limited visibility, no access to configuration, and dependence on the vendor's own governance controls.

Semantic search

Search capability that uses natural language meaning rather than keyword matching — powered by vector embeddings. In agent registries, enables discovering agents by what they do rather than requiring exact name or tag matches. Becomes essential as agent estates grow beyond a few dozen.

Shadow agent

An AI agent deployed by an individual or team without the knowledge or approval of security, compliance, or IT governance. A primary driver of agent sprawl and unmanaged risk.

Shadow AI

The unauthorized use of AI tools, agents, and services by employees without IT or security approval. Broader than shadow agents (which refers to specific deployed agents), shadow AI encompasses any unsanctioned AI usage including consumer chatbots, browser extensions, and SaaS-embedded AI features.

SOC 2

A compliance framework developed by AICPA that evaluates an organization's controls relevant to security, availability, processing integrity, confidentiality, and privacy. Increasingly applied to AI systems.

Specification failure

A failure mode in multi-agent systems where one agent misinterprets its task specification and downstream agents propagate the incorrect interpretation through the workflow. Accounts for roughly 42% of multi-agent production failures.

Stakeholder

Any person or team with an interest in an AI agent's behavior or governance — including the agent owner, security team, compliance officers, data protection officers, and business unit leaders.

Supply chain risk

Vulnerabilities introduced through third-party dependencies in an AI agent's ecosystem — model providers, API integrations, tool libraries, and frameworks. A compromised dependency can silently alter agent behavior across an entire estate.

T

Taxonomy

A hierarchical classification system used to categorize AI agents by type, function, risk level, department, or technology stack. Enables consistent governance across a diverse agent estate.

Threat detection

Real-time identification of security threats in AI agent traffic — including data exfiltration patterns, prompt injection attempts, anomalous request volumes, and model-switching attacks.

Token

The basic unit of text processed by a language model. Token counts determine API costs and are a key metric in LLM monitoring — typically measured separately for input (prompt) and output (completion).

Tool access governance

Controls that define which tools, APIs, and external systems an AI agent is permitted to use — enforced at runtime. Includes allow-lists, deny-lists, usage monitoring, and anomaly detection for tool invocations outside normal patterns.

Transparency obligation

A regulatory requirement to disclose that content was generated by an AI system, or that a user is interacting with an AI agent. Required for limited-risk systems under the EU AI Act.

Trust Service Criteria

The five principles defined by the AICPA that form the basis of SOC 2 evaluations — security, availability, processing integrity, confidentiality, and privacy. AI agent governance must map agent controls to these criteria, demonstrating how agents are secured, how data integrity is maintained, and how access is controlled.

V

Vendor lock-in

Dependence on a single AI model provider that makes it difficult or costly to switch. Agent governance platforms mitigate this by providing model-agnostic monitoring and policy enforcement.

Vendor risk management

The process of evaluating and monitoring risks introduced by third-party vendors whose products include AI agent capabilities. Extends traditional vendor risk assessment to cover agentic behavior, model governance, data handling practices, and the vendor's own AI oversight controls.

W

Workflow agent

An AI agent that orchestrates multi-step processes — calling other agents, APIs, or tools in sequence to complete a complex task. Requires governance controls at both the orchestration and individual step levels.

No matching terms found.