AI is making fast inroads into healthcare, but succeeding with these tools is often easier said than done. A major obstacle is that clinicians may not trust AI, which adds work and cost and detracts from the patient experience.
During Becker’s 16th Annual Meeting, a panel of healthcare executives discussed AI trust and deployment challenges and best practices for overcoming these barriers.
The session, titled “Trusted Clinical AI: A CXO Framework for Decision Support,” featured:
- Gregg Nicandri, MD, chief digital and innovation officer, University of Rochester Medical Center (Rochester, N.Y.)
- Gil Padula, MD, chief medical officer and market medical director, case management, St. Elizabeth Hospital and Youngstown Market, Bon Secours Mercy Health (Youngstown, Ohio)
- Harpreet Pall, MD, chief medical officer, Hackensack Meridian Jersey Shore University Medical Center (Neptune City, N.J.) and K. Hovnanian Children’s Hospital (Neptune Township, N.J.); professor of pediatrics, Hackensack (N.J.) Meridian School of Medicine
- Roy Ziegelstein, MD, Editor-in-Chief and CMO, DynaMed
Here are three key takeaways from the session:
1. Untrustworthy AI tools create real clinical and financial costs
When clinicians don’t trust AI tools they are expected to use, the result is additional “gray work” — that is, time and stress required to verify AI-generated information and make corrections.
While AI is expected to increase productivity, when it isn’t trusted by clinicians, it decreases productivity, contributes to burnout and increases costs. At the same time, patient well-being may be jeopardized, if incorrect recommendations are followed.
AI tools can provide information that’s irrelevant to patients or is based on expert opinion without evidence, Dr. Ziegelstein observed. “When a clinical decision-support model flouts the evidence and focuses on expert opinion, patients get bad care, which is costly. Patients may receive unnecessary tests and procedures,” he said.
2. Governance is key for building confidence in AI tools
Governance takes a variety of forms. First and foremost, trustworthy AI-driven healthcare tools must have a human in the loop. These humans must be experts in clinical decision support and evidence-based medicine.
In addition, both AI-tool vendors and healthcare organizations need independent AI advisory boards composed of clinicians, legal counsel, safety experts, informaticists and data privacy specialists.
According to Dr. Pall, the workflow aspect of governance is also very important. “If tools don’t fit in the workflow and are tangential to what clinical teams are doing, it will derail trust,” he said. “The tools will be ignored and you’ll be back where you started.”
3. To deploy trusted clinical AI, clinician involvement is essential
The panelists underscored the importance of adopting AI tools that are governed and designed by physicians for physicians and patients.
It’s also critical to be clear with clinicians about the problem to be solved by adopting AI. A best practice is engaging clinicians early in the decision-making process. “I want solutions that enhance the doctor-patient relationship, so doctors can have more enriched time to engage in the art of medicine,” noted Dr. Padula. “That’s a big decider.”
While it may be tempting to rush deployment of AI for fear of missing out, going slow initially often allows organizations to eventually go faster with AI. “Make sure you learn from experience and build systems in a way that enable you to do great stuff faster and in a trustworthy way by establishing principles and processes,” Dr. Nicandri said.
At the Becker's 11th Annual IT + Revenue Cycle Conference: The Future of AI & Digital Health, taking place September 14–17 in Chicago, healthcare executives and digital leaders from across the country will come together to explore how AI, interoperability, cybersecurity, and revenue cycle innovation are transforming care delivery, strengthening financial performance, and driving the next era of digital health. Apply for complimentary registration now.
