Hospitals and health systems remain cautious about using fully autonomous AI in clinical settings and instead prefer models with human oversight, according to a national survey released Dec. 16.
The November 2025 survey found that healthcare leaders consider unsupervised AI risky in high-stakes clinical environments, citing concerns about data accuracy and patient safety. Respondents consistently said AI is most effective when used to support clinicians rather than operate independently.
The survey was conducted by healthcare IT research firm Reaction Data and released by Carta Healthcare. It asked health system leaders about AI adoption, perceived risks and the role of human involvement in clinical workflows.
Here are three key findings from the survey:
- Nearly two-thirds of respondents — 62.5% — said misinterpretation of data is the top risk when AI systems operate without human oversight. Only 12.5% said fully autonomous AI has provided the most value to their organization so far.
- In contrast, 75% of respondents said they rely on human validation to ensure AI-generated outputs are clinically relevant and reliable. The same share said clinician involvement in AI design and deployment is critically important.
- Half of respondents said AI should be used to augment human decision-making rather than replace clinical tasks.
