New Healthcare AI Council Focuses on Post-Deployment Oversight to Ensure Patient Safety

The integration of artificial intelligence (AI) in healthcare has the potential to revolutionize how we diagnose and treat patients. However, with great power comes great responsibility. Waleed Mohsen, cofounder and CEO of Verbal AI Technologies, highlights a critical gap in current oversight: what happens after AI systems are deployed and start interacting with real patients at scale.

Mohsen is leading the charge to address this issue through the Council for Healthcare AI Responsibility and Safety (CHAIRS). The council's primary focus is on post-deployment monitoring, ensuring that AI systems continue to adhere to clinical protocols once they are in live use. "Healthcare puts nearly all of its AI oversight into the moment before deployment, then stops watching at the moment the risk actually starts," Mohsen said.

Bridging the Oversight Gap

Pre-deployment validation, procurement reviews, algorithm committees, and regulatory clearance are essential steps that healthcare organizations take to ensure the safety and effectiveness of AI tools. However, these processes typically assess technology at a specific point in time and do not account for how the AI performs over thousands of real-world patient encounters.

Mohsen compares this oversight gap to accrediting a hospital based on its plans before opening and then never returning to inspect actual care. "AI gets the deal no hospital would get: a careful evaluation in a controlled setting before go-live, then nothing watching actual performance," he explained. AI systems can operate at volumes far beyond human capacity, making it crucial to monitor their performance continuously.

Verbal AI Technologies, Mohsen's company, specializes in independently auditing patient interactions involving both humans and AI. This experience has informed the formation of CHAIRS, where healthcare quality, compliance, and safety leaders will shape the council's agenda, findings, and recommendations.

Why It Matters

The stakes are high when it comes to AI in healthcare. The use of AI can significantly improve productivity and the effectiveness of care delivery, allowing healthcare systems to provide better patient outcomes. However, if not properly monitored, AI systems can also introduce new risks and errors that could harm patients.

CHAIRS aims to bridge this gap by examining aggregate failure patterns from live deployments. By doing so, the council hopes to identify common issues and develop best practices for ongoing monitoring and improvement. "We need to ensure that AI systems are not only effective but also safe and ethical in their interactions with patients," Mohsen emphasized.

The public's trust in AI is essential for its widespread adoption in healthcare. As AI scales, it becomes increasingly important to demonstrate transparency and accountability. CHAIRS' efforts will play a crucial role in building this trust by providing evidence of how AI actually performs in live patient encounters, not just before implementation.

In the coming months, CHAIRS will work closely with healthcare organizations, regulators, and technology providers to establish robust post-deployment monitoring frameworks. The goal is to create a safer, more reliable healthcare environment where AI can fulfill its potential to improve patient care while minimizing risks.