As CMS and FDA Chart AI's Path Into Medicine, Clinicians Push Back on Autonomy

Imagine your doctor's office visit is covered not because a physician saw you, but because an algorithm did. That scenario is no longer hypothetical. It is the subject of active federal policymaking right now, and the decisions being made in Washington this year will shape how millions of Americans experience chronic disease care for decades to come.

At a healthcare AI event hosted by the Consumer Technology Association in Washington, D.C. on Wednesday, federal officials made clear that clinical AI has become a top strategic priority, not a side experiment. Stephanie Carlton, deputy administrator at the Centers for Medicare & Medicaid Services, laid out an agency strategy that is moving quickly on two fronts: figuring out how AI tools get approved for use, and figuring out who pays for them.

Carlton also holds a newly created title, chief clinical AI officer at CMS. The position itself is a signal. The Trump administration wants AI woven into clinical care, and it wants an official whose full-time job is making that happen.

Think of CMS's approach as resting on four legs of a table. Carlton described them as building public trust, expanding data sharing and interoperability across the healthcare system, creating clearer pathways for AI products to reach the market and be regulated, and developing frameworks for how Medicare actually pays for these tools. Remove any one leg, she suggested, and the whole strategy wobbles.

The centerpiece of that strategy is the ACCESS model, short for Advancing Chronic Care with Effective, Scalable Solutions. It's a 10-year value-based program, announced in December and launched in July, built to manage chronic conditions like diabetes, hypertension, chronic kidney disease, obesity, depression and anxiety at scale using technology and AI. More than 150 healthcare organizations were accepted into the initial launch.

Carlton hinted that CMS plans to expand the model soon, though she declined to share specifics. "We have said that we want to announce more tracks and more opportunities to engage in that, so the team has been working very carefully on those additional tracks," she said. "We're still committed to an outcomes framework for patients. We like total cost of care. We like an outcomes framework, so we'll continue that theme."

Ask her what success looks like, and Carlton points to December 2028, roughly two years from now. "I think we're hoping in December 2028 we will have seen that technology can have a massively deflationary impact on healthcare costs," she said. She also flagged something bigger than cost savings: a shift toward paying tech companies directly as Medicare providers, something ACCESS already allows. "That's a paradigm shift: paying tech companies versus just paying clinicians and healthcare facilities," she said.

The Regulatory Puzzle Ahead

Payment questions get complicated fast when the tool in question is a chatbot that can answer a health question for free. Carlton posed the dilemma bluntly: "When do we pay for it? When does that make sense? What's the difference between paying for things you can get in general apps that most of the frontier models make available quite cheaply versus what is actually performing a medical function and is reasonable and necessary for medical care, which is our standard?"

The FDA is wrestling with a parallel puzzle on the safety side. In August, the agency issued a discussion paper on how to regulate generative AI-enabled medical devices, seeking public comment through Oct. 19. The proposed framework would weigh two factors: how much autonomy a tool exercises, from simply providing information to actually taking action, and how much harm a wrong answer could cause.

The FDA is also floating a "competency-based" evaluation approach that blends laboratory-style benchmarking with real-world clinical testing. Carlton offered a helpful analogy: benchmarking is like earning a medical degree by demonstrating core competencies, while clinical validation is more like a residency, where a tool proves itself safe and effective in real conditions before it's trusted with broader responsibility.

Rick Abramson, associate director for digital health at the FDA's Center for Devices and Radiological Health, was even more direct about the mismatch between old rules and new technology. "I'm absolutely confident that the evidentiary standard for FDA authorization of generative AI tools will change because it's a square peg and round hole," he said during a later panel. He wants industry feedback on what the agency is getting right, what it's getting wrong, and how to move forward together.

Meanwhile, CMS is trying to decide what to actually measure once a tool clears the FDA's bar. Carlton said the agency is building a framework for tracking health outcomes, but the specifics depend on context: primary care versus specialty care, or a narrow clinical goal versus total cost of care. "That has implications for which health outcomes we'd be monitoring," she said. She was equally clear about wanting to avoid overcorrection. "We don't want to build a whole quality industrial complex around reporting," she said. "We want to have exactly what is necessary to make sure the technology is working and that we're not causing harm."

Behind the policy language sits a genuinely contested question: how much should a machine be allowed to decide on its own?

Jesse Ehrenfeld, past president of the American Medical Association and now global chief medical officer at Aidoc, offered a grounded example. Even something as routine as an AI-managed prescription refill, currently being piloted in Utah, can hit a wall that only human judgment can clear. "I've had patients walk in the office, and you know something's not right," he said. "There's a tremor. There's something else going on, and that's when you execute your cognitive function." Some context, he argued, simply cannot be fed into an autonomous system.

Marc Paradis, principal at SIYOM Consulting, pushed the conversation forward rather than backward, arguing AI's next generation will detect things humans physically cannot: subtle voice changes, tremors, physiological signals invisible to the naked eye. He pointed to hyperspectral imaging during surgery as one example of AI spotting patterns no surgeon could see. "That world will come, and it will come much faster than we think," he said.

John Whyte, CEO of the American Medical Association, pushed back hard on that framing. He argued human and machine intelligence are fundamentally different things, and that enthusiasm should not outrun evidence. "What we should be talking about is what is the evidence base that we need to make decisions as it relates to safety, as it relates to patient outcomes," Whyte said, adding that AI deserves the same evidentiary rigor as any other medical intervention.

Laura Adams, senior advisor at the National Academy of Medicine, challenged a different assumption altogether: that a clinician must always review every AI decision. In areas like image analysis, where AI has already shown strong performance, mandatory human review can slow care down without meaningfully improving it. The better question, she said, is figuring out exactly when clinician involvement helps and when it gets in the way.

Why It Matters

None of this is abstract for patients managing diabetes or depression today. The frameworks being written right now will determine whether AI tools reach them safely, whether insurance actually covers those tools, and whether a human being remains available when something doesn't look right. Getting the balance wrong in either direction, moving too slowly or trusting automation too readily, carries real consequences for real people. The debate happening in conference rooms this week is really a debate about how much we trust machines with the most consequential moments in our health, and that answer deserves as much scrutiny as the technology itself.