A pilot study led by researchers at UC San Diego Health found an advanced AI system using LLMs achieved 90% agreement with manual quality reporting measures. The team tested the system on a sample of 100 abstractions for the CMS SEP-1 measure for severe sepsis and shock, finding the model achieved agreement with manual abstractions in 90 of the 100 abstractions included in the study.
The findings suggest that LLMs can perform accurate abstractions for complex quality measures that typically require meticulous analysis of patient charts. The abstraction process for SEP-1, for example, typically involves a 63-step evaluation that requires weeks of effort from reviewers.
“We remain diligent on our path to leverage technologies to help reduce the administrative burden of healthcare and, in turn, enable our quality improvement specialists to spend more time supporting the exceptional care our medical teams provide,” Chad VanDenBerg, study co-author and chief quality and patient safety officer at the health system, said in a news release.
At the Becker's 11th Annual IT + Revenue Cycle Conference: The Future of AI & Digital Health, taking place September 14–17 in Chicago, healthcare executives and digital leaders from across the country will come together to explore how AI, interoperability, cybersecurity, and revenue cycle innovation are transforming care delivery, strengthening financial performance, and driving the next era of digital health. Apply for complimentary registration now.
