Hospitals Can't Wait for AI Rules: Why Governance Must Start Now

Imagine a family, months after a difficult diagnosis, asking a hospital to explain exactly how an AI tool shaped their loved one's care. Which version of the software wrote the notes. Whether anyone reviewed them. Whether the patient even knew a machine was listening in the room. For most health systems today, answering those questions with confidence is not yet possible. That gap is the real story behind the rapid rollout of generative AI in clinical settings.

Attorneys and governance experts are blunt about what this means. The absence of a finished regulatory rulebook is not a reason to slow down. It is the reason to move faster on building internal safeguards, because when the rules eventually arrive, hospitals without a paper trail will be exposed.

The challenge goes well beyond the narrower question of whether an AI-generated note belongs in a patient's chart. Health systems now need governance covering who owns each AI-generated document, how long it gets kept, how software versions get tracked over time, how audits and legal holds get handled, how patient consent gets managed, and what happens when a vendor changes its product after go-live.

Thomas F. O'Neil III, a managing director at research and consulting firm BRG, frames AI deployment as both a growth strategy and a risk-management duty that belongs at the board level. He argues senior leaders need real controls: written policies, staff education, ongoing compliance monitoring, and a clear path to corrective action when something goes wrong. Cross-disciplinary committees can run the daily work, he says, but the board itself must stay informed about material risks and shifting legal requirements.

"The health system owns, and is responsible for, the clinical record, regardless of what third-party tool produced the draft," O'Neil said. Every AI-generated document, in his view, needs a named human owner inside the organization, not just a vendor's logo attached to it.

On retention, O'Neil recommends a written schedule that separates final, signed clinical records from transitional material like raw audio recordings and draft notes. The signed record should follow the retention rules that already apply to medical records generally. Transitional material can follow a shorter schedule, but O'Neil stresses that hospitals need to document their reasoning and then apply it consistently, not case by case.

Tracking a Moving Target

Version control might be the hardest piece of this puzzle, and also the most consequential. AI vendors regularly update their models, adjust prompts, and tweak configurations, sometimes changing how the same clinical input gets handled a few months later. That creates a real problem if a hospital later faces a complaint, a lawsuit, or a regulatory inquiry. Investigators will not want to know what today's version of the tool would produce. They will want to know what it actually produced during that specific patient encounter, at that specific moment.

"When an AI tool updates its model or changes its logic, the hospital or health system must be able to reconstruct what the tool produced at the time of a given patient encounter," O'Neil explained. Doing that requires tracking model versions, prompt versions, and any configuration changes that affect output, along with contract language requiring vendors to notify customers when something material shifts.

Audit trails need to capture the full chain of custody, from the moment AI generates content through a clinician's review and final decision to accept or reject it. Legal and compliance teams need the ability to place relevant material under legal hold quickly, before it disappears into a routine deletion cycle. O'Neil suggests quarterly reporting to the board or an appropriate committee, given how fast the underlying technology keeps changing.

Consent is its own layer of complexity. O'Neil points to lawsuits filed in April 2026 alleging that ambient AI scribes recorded physician-patient conversations without proper consent, then routed audio and transcripts to third-party servers. A subtler version of the same problem: a patient consents to an AI scribe in the exam room, but a family member sitting nearby never did, which can trigger separate privacy or state recording-law issues nobody planned for.

Human oversight alone will not close that gap. O'Neil describes a widening "human-in-the-loop" problem, where AI deployment is outpacing the governance capacity meant to supervise it. Clinician review only works, he says, when it is backed by real policy, proper implementation, and active monitoring, not treated as a rubber stamp.

Jim Flynn and Alaap Shah, healthcare attorneys at Epstein Becker Green, recommend a governance committee at the board or C-suite level bringing together legal, compliance, clinical, IT, and patient-safety leadership. Its job should span vendor selection, validation, ongoing monitoring, and incident response. "AI adoption is not treated as a procurement decision," Flynn said. "As I see it, it's a governance decision with clinical, legal and operational implications."

Shah urges hospitals to independently validate AI performance across diverse patient populations rather than take vendor claims at face value, document that testing, and set clear guardrails for acceptable use before deployment even begins. Contracts should include audit rights, access to a vendor's own validation and performance data, advance notice of model updates, and firm restrictions on using patient data to train future models.

Once that framework exists, retention should separate three categories: clean, clinician-reviewed clinical documentation; validation studies, performance metrics, bias testing, and vendor compliance records kept ready for review; and protected internal material such as risk assessments and vendor communications, walled off in a restricted repository. Shah also recommends periodic audits comparing AI-suggested content against what clinicians actually finalized, watching for accuracy problems or embedded bias, alongside a defined incident-response process for escalation, investigation, and any required regulatory notification.

Why It Matters

Neither federal rules nor the growing patchwork of state laws offers a complete roadmap for generative AI in healthcare right now. Shah's advice is to plan for the strictest plausible future rather than today's minimum requirements. "Do not wait for regulatory clarity because the regulatory landscape for AI in healthcare is a work in progress," he said. "Build your governance framework now as if the most stringent requirements will apply."

O'Neil expects signed AI-assisted notes will eventually be treated much like any other clinical record. But the provenance behind that note is genuinely new: an outside vendor, a model trained on vast datasets, shifting configurations, and a clinician who ultimately signs off. AI's reach now extends into revenue cycle, finance, and legal operations too, multiplying the number of places where retention and accountability rules can quietly diverge.

The goal is not to preserve every scrap of data an AI system ever generates. It is to be able to explain, years later if necessary, why an organization kept what it kept, deleted what it deleted, and knew exactly who was accountable for the outcome. As Flynn puts it, a coherent, well-documented governance framework can become an organization's strongest defense the day regulators or plaintiffs' lawyers come knocking.