Agent governance in healthcare: EU AI Act, EHDS and the Nordic compliance stack
David Baum 18 min read The most governed industry meets the least governed technology#
Healthcare has always been the most heavily regulated domain for technology. Every system that touches patient data, clinical decisions or care delivery operates under layers of compliance obligations: GDPR, the EU Medical Device Regulation, national health data laws and professional codes of conduct that date back centuries.
AI agents are about to collide with all of them simultaneously.
The collision is already producing casualties. ECRI, the global healthcare safety nonprofit, named AI in healthcare the number one health technology hazard for both 2025 and 2026. Their president, Marcus Schabacker, put it directly: “Balancing innovation in AI with privacy and safety will be one of the most difficult, and most defining, endeavors of modern medicine.”
For organizations building agent inventories today, healthcare is where the governance gap is widest, and where the consequences of that gap are most severe.
Gravitee’s State of AI Agent Security 2026 report found that among organizations with active AI agent deployments, 88% reported confirmed or suspected security incidents in the past year. In healthcare, that number climbs to 92.7%, the highest of any industry. An academic analysis of 82 health-related AI incidents found that 18% had high or critical severity, including cases where AI algorithms delayed organ transplants and caused errors in patient allocation.
This article maps the European regulatory stack for healthcare AI agents (the EU AI Act, GDPR, the European Health Data Space, the Medical Device Regulation and NIS2), explains why autonomous agents create governance challenges that traditional healthcare IT never faced and provides a practical framework for Nordic and European healthcare organizations navigating this convergence.
The European healthcare AI regulatory stack#
Healthcare organizations deploying AI agents in Europe don’t face one regulation. They face five, converging simultaneously.
EU AI Act (high-risk requirements from August 2026). The Act classifies several healthcare AI use cases as high-risk under Annex III: AI evaluating eligibility for healthcare services (5a), AI for risk assessment and pricing in life and health insurance (5c) and AI for emergency triage and dispatch classification (5d). Any AI system that qualifies as a medical device under the MDR and requires third-party conformity assessment is automatically high-risk under Article 6(1). A PMC analysis of the Act’s healthcare implications notes that approximately 75% of all commercial AI-enabled medical devices are in radiology, and almost all are classified as Class IIa or above under the MDR, meaning most will be automatically classified as high-risk under the AI Act.
GDPR (in force since 2018, but with new AI-specific enforcement). Every healthcare AI agent that processes patient data is processing personal data, and often special category data (health data) under Article 9. The European Data Protection Supervisor published orientations in 2025 on applying data protection principles to generative AI, reinforcing that purpose limitation, data minimization and transparency obligations apply fully to AI systems.
European Health Data Space (EHDS, regulation applies from March 2026). The EHDS creates a framework for cross-border health data exchange and secondary use. EHR systems must expose standardized APIs. National Health Data Access Bodies will govern secondary use permits. This is the data infrastructure that healthcare AI agents will increasingly connect to, and governance of agent access to EHDS data will become a compliance requirement. The EHDS is expected to generate 11 billion euros in savings over the next decade by enhancing data accessibility, but only if the governance layer keeps pace.
EU Medical Device Regulation (MDR). AI that qualifies as Software as a Medical Device (SaMD) must undergo conformity assessment, CE marking and post-market surveillance. Clinical decision support tools, diagnostic AI and triage systems are all potentially in scope. The MDR and the AI Act apply in parallel, meaning healthcare AI providers face dual compliance obligations.
NIS2 Directive (enforcement timelines vary by Nordic country). NIS2 imposes cybersecurity and incident reporting obligations on essential service providers, including healthcare. As Twoday’s Nordic AI governance analysis details, the Nordic enforcement timeline is staggered: Denmark’s NIS2 has been mandatory since July 2025, Sweden’s entered force in January 2026, Finland’s AI Act supervision became active in January 2026 and Norway’s AI Act implementation is expected in summer 2026.
No healthcare sector has faced this many overlapping AI regulations at once. Twoday’s Alina Negulescu argues that Nordic organizations should build a single unified governance infrastructure (one AI system inventory covering risk tier, sector, data categories, supplier and purpose) that satisfies NIS2, GDPR, the AI Act and ISO 42001 simultaneously, rather than maintaining four separate compliance programs.
Why healthcare agents are different#
Healthcare AI agents don’t just process data. They access the most sensitive category of personal data that exists (health records), they operate in environments where errors can cause physical harm or death and they interact with systems (EHRs, pharmacy systems, lab information systems) that are deeply embedded in clinical workflows.
Three characteristics make healthcare agents uniquely challenging to govern.
The decision authority spectrum#
Not all healthcare agents carry the same risk, and the governance requirements should reflect this. A scheduling agent that books patient appointments operates in a fundamentally different risk category than a clinical decision support agent that recommends treatment protocols.
| Agent type | Decision authority | Data access | Risk tier |
|---|---|---|---|
| Scheduling and booking | Read-only, administrative | Minimal (name, appointment) | Low |
| Billing and claims processing | Execute with business rules | Moderate (diagnosis codes, dates of service) | Medium |
| Clinical documentation and transcription | Generate content, human reviews | High (full clinical encounter notes) | High |
| Triage and patient routing | Recommend priority, human acts | High (symptoms, vitals, history) | High |
| Clinical decision support | Recommend treatment, clinician acts | Full (complete patient record) | Critical |
| Autonomous treatment decisions | Act with minimal oversight | Full | Critical |
The EU AI Act’s Annex III reinforces this spectrum. Emergency triage systems are explicitly listed as high-risk (Recital 58 explains that these “make decisions in very critical situations for the life and health of persons”). Insurance risk assessment and pricing for life and health insurance are high-risk under Annex III point 5(c). But a scheduling agent that doesn’t materially influence care decisions may qualify for the Article 6(3) derogation as a “narrow procedural task.”
The governance framework must classify each agent on this spectrum and calibrate oversight accordingly, not apply the same controls to a billing bot and a diagnostic agent.
The minimum necessary principle, reimagined for agents#
One of GDPR’s foundational principles (and a longstanding healthcare governance requirement) is data minimization. Agents should access only the data they need for their specific function.
This principle, already well-established in European healthcare, takes on new urgency with AI agents. An agent that processes insurance claims needs diagnosis codes and dates of service. It does not need the patient’s therapy notes. A transcription agent needs the audio of a clinical encounter. It does not need the patient’s full longitudinal record.
But AI agents, by their nature, are often configured with broad access to perform well across varied scenarios. The temptation is to give an agent read access to the entire EHR “just in case.” This is precisely what data minimization prohibits, and it’s where most healthcare organizations will face their first governance challenge with agents.
Every healthcare agent needs a data access scope that is documented, justified, technically enforced (not just policy-stated) and auditable. The agent registry must capture what data each agent can access, and the monitoring layer must verify that access stays within those boundaries.
The audit trail as a liability shield#
In European healthcare, the audit trail is not just a compliance requirement. It is a legal defense.
The revised EU Product Liability Directive, set to be transposed into member-state law by December 2026, removes the previous 85 million euro liability cap for personal injury from AI systems. Latent injury liability extends up to 25 years. And in cases involving “black-box” AI where causation is difficult to prove, courts can presume the product was defective unless the provider demonstrates otherwise.
If an agent contributes to a diagnostic error, a treatment delay or a patient harm event, the provider’s audit trail becomes the primary evidence in litigation. The trail must show exactly what data the agent accessed, what reasoning chain it followed, what tools it invoked and what human oversight was available. If the audit trail is absent, incomplete or fragmented, the court can presume defectiveness. Latent injury liability extends up to 25 years under the revised Product Liability Directive.
For healthcare organizations, this shifts the governance calculation. An incomplete audit trail is no longer a compliance gap. It is a legal vulnerability. Every agent interaction with clinical data must be logged with enough detail to reconstruct the decision chain years after the event. The organizations investing in this infrastructure now are building their litigation defense in advance, not after an incident.
The identity layer is equally critical. Healthcare AI agents often inherit the access credentials of the clinician who deployed them, creating a shadow permission structure that no CISO can audit. When an agent acts on behalf of a physician, who is accountable: the physician, the agent’s developer or the organization?
Dr. Sean Kelly, chief medical officer at Imprivata and an emergency physician at Beth Israel Deaconess Medical Center, captured the identity governance challenge at HIMSS26:
If you have agentic agents doing things in healthcare, it’s a complex and messy situation… The problem is that leaves you very vulnerable because it’s even harder than human identity to control and to understand.
The five governance requirements that hit healthcare agents hardest#
1. Agent identity and access controls#
Every healthcare AI agent needs a unique, verifiable identity, the same governance rigor applied to human users accessing clinical systems. This means individual credentials (not shared API keys), role-based access scoped to the agent’s specific function and audit-logged authentication for every system interaction.
The AI Act’s Article 14 requires that high-risk AI systems be designed for effective human oversight, including the ability to identify the system and its actions. NIS2 requires identity management for systems accessing essential service infrastructure. GDPR requires that data processing be traceable to specific processors and purposes.
For healthcare specifically, agent identities must integrate with existing clinical identity management. The same systems that govern which physicians can access which patient records must extend to govern which agents can access which data. An agent operating in a radiology workflow should not have credentials that grant access to behavioral health records.
2. Immutable audit trails#
The EU AI Act’s Article 12 requires automatic event logging for high-risk AI systems throughout their lifetime. For healthcare agents, this means logging every clinical data access, every tool invocation, every recommendation generated and every handoff to a human clinician, with sufficient detail to enable post-hoc assessment of any decision.
The academic framework for auditable clinical AI published in 2026 proposes a tamper-evident audit logging mechanism that records the original query, patient data elements accessed (stored as cryptographic hashes for privacy), unique identifiers of retrieved evidence sources and the specific reasoning steps taken. This level of traceability isn’t theoretical aspiration. It’s what the Product Liability Directive will make discoverable in litigation.
Dr. Ed Lee, chief medical officer at Nabla, stressed the tiered governance imperative at HIMSS26:
Especially as we’re talking and moving into things like agentic AI, where things are potentially happening in an autonomous way, there may be even further guardrails so that we don’t do the wrong thing. We always have the patient’s wellbeing in mind and safety and quality are of most importance.
3. Human oversight with clinical accountability#
The AI Act’s Article 14 requires that humans can understand the system’s output, override or reverse it and halt the system through a stop mechanism. In healthcare, this translates to a specific requirement: the clinician must retain clinical accountability for every decision, even when an AI agent contributed to the reasoning.
This creates a design constraint for healthcare agents. The agent must present its recommendations in a way that the clinician can critically evaluate, not just accept. Confidence intervals, flagged uncertainties and highlighted edge cases are not optional UX features; they’re governance requirements.
The Two Birds liability analysis notes that human-in-the-loop doesn’t automatically shield the AI provider from liability. Manufacturers must foresee clinician errors (alarm fatigue, cognitive overload during high-workload shifts, misinterpretation of AI outputs) and design safeguards accordingly. An audit trail that shows the clinician “approved” the AI’s recommendation doesn’t protect the provider if the approval interface was designed in a way that discouraged critical review.
4. Continuous post-market monitoring#
The AI Act’s Article 72 requires providers of high-risk AI systems to establish post-market monitoring that actively and systematically collects data on performance throughout the system’s lifetime. For healthcare agents, this means monitoring not just technical performance (latency, error rates) but clinical performance: are the agent’s recommendations accurate? Are they consistent across patient populations? Is performance degrading over time?
This is where traditional MLOps monitoring falls short. An agent’s model-level metrics might be stable while its clinical recommendations drift, because the data environment changed, because a guideline was updated or because the patient population shifted. Healthcare agents require behavioral monitoring that tracks the quality of clinical outputs, not just the performance of underlying models.
The MDR adds a parallel requirement: post-market surveillance for any agent classified as a medical device. The surveillance must detect safety issues, track complaint trends and trigger corrective action when problems emerge.
5. Data governance across the EHDS#
As the European Health Data Space comes into force, healthcare agents will increasingly access data through standardized APIs and cross-border exchange mechanisms. Every access must comply with GDPR principles, EHDS permit conditions and national health data laws.
The governance challenge is that EHDS creates new data flows that didn’t exist before. An agent in a Danish hospital might access patient data from a Swedish EHR through the EHDS infrastructure. Who governs that agent’s access? The Danish Health Data Access Body, the Swedish one, or both? The regulatory answer is still being worked out, but the governance infrastructure must be ready.
The Nordic context: converging compliance, diverging timelines#
The Nordics are uniquely positioned for healthcare AI governance, and uniquely exposed to its challenges. Nordic countries have the most digitized healthcare systems in Europe, the strongest public health data registries and the highest rates of AI adoption. They also face the most compressed compliance timeline.
Twoday’s analysis maps the convergence: Denmark’s NIS2 is already mandatory. Sweden’s NIS2 entered force in January 2026, and uniquely among the Nordics, Sweden can impose fines on both public and private entities, and management may face prohibitions on holding management functions. Finland became the first Nordic country with active AI Act supervision in January 2026. Norway’s AI Act implementation is expected in summer 2026.
The practical reality is that Nordic healthcare organizations need to comply with the AI Act, NIS2, GDPR and in many cases ISO 42001, all at once. Building four separate compliance programs is unsustainable. The organizations that will succeed are those building unified governance infrastructure: one agent registry, one risk classification framework, one audit trail, one certification workflow, covering all four frameworks simultaneously.
The FLORENCE project demonstrates what cross-border Nordic healthcare AI governance looks like in practice. This three-year collaboration between Denmark, Norway and Sweden developed federated learning infrastructure for colorectal cancer treatment using the OMOP data model. The project produced a joint Nordic strategy and national action plans, solving data harmonization, privacy-preserving computation and cross-border audit trail challenges along the way. It’s a model for how clinical AI can be governed across jurisdictions, but it required deliberate governance architecture from the start.
AI Sweden’s healthcare initiative, funded by Vinnova, mapped all AI initiatives across Swedish healthcare regions through its Vardkartan (Healthcare Map). The findings confirm what the governance gap implies: the majority of initiatives are in diagnostics and administration, but governance maturity varies dramatically across regions.
Fredrik Linden, founder of Hamling and contributor to the MyData Nordic Hub, captured the Nordic imperative in a February 2026 analysis:
Healthcare AI is scaling faster than our ability to trust it. The Nordics can help set that standard globally, if we insist on governance that works where it matters most: from the point of care to the board room.
His proposed Nordic trust model spans six layers (from machine-readable consent to evidence-based AI standards) built on existing Nordic infrastructure like Finland’s Kanta and Sweden’s 1177/NPO.
How Roval implements healthcare agent governance#
Roval is the enterprise system of record for AI agents, and the platform’s architecture maps directly to the healthcare governance requirements described in this article.
Agent registry with healthcare risk classification. Every agent is registered with full identity, ownership and technical metadata. The risk classification includes a data sensitivity dimension: agents accessing protected health information are automatically classified as Tier 3 (High) or above. Agents accessing health data through EHDS interfaces are flagged for cross-border governance requirements. The dependency graph maps agent connections to EHR systems, lab information systems and pharmacy platforms.
Unified compliance certification. Certify agents against GDPR, the EU AI Act and custom healthcare-specific frameworks (including MDR post-market surveillance requirements) from a single workflow. Per-requirement evidence tracking with document uploads, URL references and attestation text. Auto-expiry by risk tier: 90 days for Critical (clinical decision support, triage), 180 days for High (documentation, transcription), 365 days for Low (scheduling, administrative). One certification workflow satisfies multiple regulatory frameworks simultaneously, exactly the unified approach that Nordic compliance requirements demand.
Immutable audit trail for litigation readiness. Every agent action, data access, tool invocation and decision point recorded with actor, timestamp and before-and-after state. The trail is append-only, readable by governance and compliance teams but not modifiable by engineering teams. Exportable as CSV or JSON, filtered by agent, date range, action type or resource, ready for regulatory inquiry, audit or litigation discovery. This is Article 12 compliance and Product Liability Directive readiness in a single system.
Continuous monitoring with clinical safety signals. The Observer captures every tool call in real-time. The LLM proxy captures every prompt sent to LLM APIs with under 1ms of overhead. Policy rules evaluate compliance within 30 seconds. Drift detection runs every 15 minutes, catching certification expiry, configuration changes, model updates and owner departures. The circuit breaker auto-stops agents that exceed violation thresholds, the kill switch that Article 14 requires.
Production gate enforcement. Critical and high-risk healthcare agents cannot transition from staging to production without active, non-expired certification against all applicable frameworks. This is not a guideline. It’s a hard technical block with an explanatory error. The gate ensures that no uncertified clinical agent reaches the systems where it can affect patient care.
Sources and further reading#
More in governance
The lethal trifecta: governing the three capabilities you can't remove
Five days in January 2026, four AI productivity tools shipped indirect prompt injection vulnerabilities. Same pattern in every case. Simon Willison named it in June 2025: private data access plus untrusted content plus external communication equals data exfiltration. You can't remove any leg without breaking the agent. Governance has to shift from prevention to containment.
David Baum The confused deputy problem: lessons from Meta's AI agent Sev-1
On March 18 2026, a Meta AI agent exposed restricted company and user data for two hours. The credentials were valid. The governance was not.
David Baum Guardian agents: when AI governs AI
Humans can't watch every agent, every action, every second. Guardian agents (dedicated policy engines that monitor, audit and enforce governance on other agents at runtime) resolve the tension between machine-speed execution and meaningful oversight.
David Baum