Hospitals and health systems remain cautious about using fully autonomous AI in clinical settings and instead prefer models with human oversight, according to a national survey released Dec. 16.
The November 2025 survey found that healthcare leaders consider unsupervised AI risky in high-stakes clinical environments, citing concerns about data accuracy and patient safety. Respondents consistently said AI is most effective when used to support clinicians rather than operate independently.
The survey was conducted by healthcare IT research firm Reaction Data and released by Carta Healthcare. It asked health system leaders about AI adoption, perceived risks and the role of human involvement in clinical workflows.
Here are three key findings from the survey:
- Nearly two-thirds of respondents — 62.5% — said misinterpretation of data is the top risk when AI systems operate without human oversight. Only 12.5% said fully autonomous AI has provided the most value to their organization so far.
- In contrast, 75% of respondents said they rely on human validation to ensure AI-generated outputs are clinically relevant and reliable. The same share said clinician involvement in AI design and deployment is critically important.
- Half of respondents said AI should be used to augment human decision-making rather than replace clinical tasks.
Becker’s 3rd Annual Fall Chief Pharmacy Officer Summit is the top tier event that the people leading the future of pharmacy convene. This 3-day event is built specifically for Chief Pharmacy Officers and senior pharmacy leaders. The sessions examine the workforce, financial and clinical priorities shaping pharmacy today. Becker’s is where candid discussions flourish and new ideas thrive. Becker's 3rd Annual Fall Chief Pharmacy Officer Summit
