Study finds hallucinations in hospital-used AI tool

About 1% of audio transcriptions written by Open AI’s Whisper, a speech recognition tool launched in 2022 and currently used in hospitals, contain hallucinated phrases or sentences when nobody spoke, according to a recent study

Advertisement

Researchers at Cornell University in Ithaca, N.Y., University of Washington in Seattle, New York University in New York City and University of Virginia in Charlottesville investigated how aphasia, which often appears after strokes, could affect Whisper-made transcriptions. 

More than 85 health organizations and 45,000 clinicians are using a Whisper-based tool built by Nabla.

The researchers fed Whisper about 40 hours of data, including 23 hours of speech from people with aphasia. Of the 1% of hallucinated text, 38% of it featured a perpetuation of violence, inaccurate associations or false authority. 

The findings were presented in the Association for Computing Machinery’s “FAccT ’24: The 2024 ACM Conference on Fairness, Accountability and Transparency,” held in Rio de Janeiro June 3-6.

At the Becker's 11th Annual IT + Revenue Cycle Conference: The Future of AI & Digital Health, taking place September 14–17 in Chicago, healthcare executives and digital leaders from across the country will come together to explore how AI, interoperability, cybersecurity, and revenue cycle innovation are transforming care delivery, strengthening financial performance, and driving the next era of digital health. Apply for complimentary registration now.

Register to Attend Webinar

The 2 KPIs that drive readmissions — and how hospitals act on them in real time

Tuesday, August 11
1:00 PM - 2:00 PM CDT

Presenter: Nik Rao, Dexur

Advertisement

Next Up in Patient Safety & Outcomes

Advertisement

Comments are closed.