AI Drug Diversion Tools Struggle Without Human Oversight in Hospitals

In late September 2024, patients at Adventist Health in Bakersfield, California, noticed something was off about a nurse. She was walking barefoot through the intensive care unit, talking to herself, and acting abrasively toward others. Family members observed her sloppily removing IV needles but were too afraid to confront her. One patient, suffering from severe pain, realized he wasn’t getting the medications he needed.

The nurse, hired through a travel nursing agency just weeks before, was stealing fentanyl and morphine from a secured cabinet in the unit. She would document that she had administered these drugs to patients, but instead, she was using them herself. The Centers for Medicare and Medicaid Services (CMS) investigated the incident after receiving complaints.

This case is not an isolated incident. Hospitals are repositories of addictive substances that are also vital medical treatments. Employees sometimes steal these drugs, leading to serious patient safety issues. New technologies, such as machine learning software, aim to identify theft and connect individuals with substance abuse resources. However, the Adventist Health incident shows that AI alone isn’t enough; human oversight and trust in the system are crucial.

The Role of AI in Drug Diversion

Tools like ControlCheck by Sentri7 use advanced algorithms to monitor drug usage patterns and flag suspicious activities. These systems can detect when a nurse or other staff member is accessing medications more frequently than necessary or at unusual times. In theory, this should help hospital administrators catch potential drug diversion early.

However, the effectiveness of these tools depends heavily on how they are used. At Adventist Health, the machine learning software had already flagged the nurse’s behavior as suspicious, but hospital managers ignored the alerts. According to auditors, the lack of action was due to a combination of factors: inadequate staff training, a lack of trust in the AI warnings, and a failure to follow established protocols.

Dr. Sarah Thompson, a public health researcher specializing in hospital safety, explains, “AI can provide valuable insights, but it’s only as good as the humans who use it. If staff are not trained to recognize and act on these alerts, the technology is useless.”

The consequences of ignoring AI warnings can be severe. In the Adventist Health case, patients suffered unnecessarily because they were not receiving the medications they needed. The nurse’s behavior also put other patients at risk of infection or further harm.

What Comes Next

To improve drug diversion detection and prevention, hospitals need to invest in comprehensive training programs for staff. These programs should emphasize the importance of AI alerts and provide clear guidelines on how to respond to them. Building trust in the technology is crucial. Staff must understand that these systems are designed to protect both patients and employees.

Dr. Thompson adds, “It’s about creating a culture of safety where everyone feels responsible for patient well-being. This includes addressing the root causes of substance abuse among healthcare workers, which can be complex and multifaceted.”

Cybersecurity is another critical aspect to consider. As hospitals increasingly rely on digital systems to manage drug inventories and track usage, the risk of cyberattacks also rises. Hospitals must implement robust security measures to protect sensitive data and ensure that AI tools are not compromised.

The Adventist Health incident serves as a stark reminder that while technology can enhance patient safety, it is not a substitute for human judgment and ethical responsibility. By combining advanced AI with diligent human oversight, hospitals can better protect their patients and staff from the dangers of drug diversion.