At the Becker’s Annual Meeting, Elsevier’s ClinicalKey AI sponsored a roundtable that drew clinical and informatics leaders from health systems across the U.S. to examine a central question shaping healthcare AI adoption:
Is AI earning its place in clinical decision‑making — or introducing new forms of risk, friction, and uncertainty?
The answer, according to participants, is nuanced. While AI adoption is accelerating, success is not determined by technical capability alone. Instead, trust, usability, governance, and clinical alignment are emerging as the true differentiators between tools that scale and those that stall.
What healthcare leaders need to know
AI adoption is advancing but trust is lagging
Across organizations, AI is gaining traction, particularly in well-defined clinical domains such as radiology, cardiology, and oncology. These use cases share a common thread: they are narrow in scope, clinically owned, and clearly integrated into workflows.
By contrast, broad or generalized AI tools—especially those passively embedded into electronic health records—are often met with skepticism. Leaders report that these tools can be harder to validate, harder to trust, and more difficult to justify clinically and financially.
“Human-in-the-loop” is essential but not guaranteed
Leaders consistently emphasized concern about automation bias and the erosion of clinical reasoning skills, particularly as AI becomes integrated earlier in training and daily practice.
Several organizations are already monitoring clinician behavior (e.g., whether AI‑generated content is accepted without modification), but sustainable approaches to preserving clinical judgment remain unclear.
Trust is earned through transparency, not explainability alone
Participants repeatedly distinguished between explainability (“how the model works”) and Operational ptransparency (“when I should trust it, when I shouldn’t, and who owns it”).
Health systems are already implementing measures such as:
- Visual indicators that label AI-generated insights
- Defined confidence thresholds and “no-score” outputs
- Documented model ownership and governance structures
Importantly, trust is fragile: even a single poor clinical experience can significantly undermine confidence in AI tools.
Governance has become a front‑line adoption issue
Most health systems now have some AI governance frameworks in place, including review boards, intake processes, and vendor risk assessments. Still, the burden is growing.
A consistent principle has emerged: AI does not make decisions—clinicians do. Accountability remains firmly with licensed providers, while governance bodies focus on evaluating and mitigating risk.
Leaders are increasingly looking for partners who can simplify governance and provide credible support in risk assessment, not just deliver technology.
Adoption depends more on enablement than on mandates
Top-down directives to “use AI” are largely ineffective. Instead, successful organizations focus on:
- Empowering clinical champions to validate and advocate for tools
- Facilitating peer-to-peer learning (e.g., testimonials, forums)
- Building AI literacy at both the clinical and executive levels
- Leveraging informatics teams as translators between technology and care delivery
The takeaway: Adoption is a cultural and educational journey, not a technology rollout.
Data quality remains the silent constraint
Participants were blunt: clinical data is fragmented, duplicative, and often unreliable. AI amplifies this reality.
Issues such as conflicting medication lists or duplicated records can undermine AI outputs, reinforcing the principle of “garbage in, garbage out.”
However, when designed thoughtfully, AI can also improve documentation quality by creating feedback loops that help clinicians correct and refine data over time.
ROI must go beyond cost savings
Many AI tools today do not deliver immediate financial returns and may even increase operational costs due to governance and oversight requirements.
Yet healthcare organizations are increasingly willing to invest in AI that demonstrates:
- Improved clinical outcomes
- Enhanced patient safety
- Reduced risk and liability
- Competitive differentiation for recruitment and reputation
In some cases, alternative funding sources (such as foundations or endowments) are supporting AI initiatives when value can be clearly articulated beyond efficiency gains.
Key takeaways
- Trust is the new currency: AI adoption hinges more on credibility and safety than on innovation alone
- Start narrow to scale effectively: Focus on clearly defined, clinically owned use cases
- Protect clinical judgment: Ensure AI tools reinforce—not replace—critical thinking
- Prioritize transparency: Make it clear when and how AI should be used, and who is accountable
- Strengthen governance early: Robust oversight is essential and increasingly expected
- Invest in enablement: Education, champions, and peer learning drive adoption more than mandates
- Fix the data foundation: High-quality, reliable data is essential for meaningful AI impact
- Redefine ROI: Measure value in terms of outcomes, safety, and risk; not just cost savings
As healthcare continues to navigate the evolving role of AI, one theme stands out: the most successful solutions will be those that strengthen—not substitute—clinical judgment, while earning trust at every step of the care journey.
To see what this looks like in practice, explore how ClinicalKey AI is designed to support clinicians with trusted, evidence-based insights, transparent outputs, and workflows that reinforce clinical expertise. To learn more, visit www.elsevier.com/clinicalkey-ai.
At the Becker's 11th Annual IT + Revenue Cycle Conference: The Future of AI & Digital Health, taking place September 14–17 in Chicago, healthcare executives and digital leaders from across the country will come together to explore how AI, interoperability, cybersecurity, and revenue cycle innovation are transforming care delivery, strengthening financial performance, and driving the next era of digital health. Apply for complimentary registration now.
