EHR vendor Epic's AI-powered sepsis model, which is used by hundreds of hospitals in the U.S., has faced scrutiny, CNBC reported April 3.
Epic's model is used to predict sepsis, a potentially life-threatening bloodstream infection. But in 2023, a study published in JAMA found that the tool missed a higher share of true cases and was less timely than other sepsis tools.
According to the researchers who conducted the study, Epic's model was found to be more accurate than other models at higher threshold prediction, but it missed a higher share of true cases and was less timely than the Systemic Inflammatory Response Syndrome tool and the Sepsis-Related Organ Failure Assessment tool.
In June 2022, Epic told Becker's it had made changes to its sepsis prediction model in a bid to improve its accuracy and make its alerts more meaningful to clinicians.
"Last fall, we released an updated version of the sepsis predictive model and we are working with our customers to implement it. The live organizations have seen more timely alerts and fewer false positives. The JAMA study does not reflect the performance of our updated model," an Epic spokesperson told Becker's.
In February 2024, another study conducted by researchers at Ann Arbor-based University of Michigan claimed that the model's software struggles to differentiate between high- and low-risk patients before they receive treatments. The University of Michigan study used the older version of Epic's sepsis model.
The researchers found that the Epic model correctly identified high-risk patients 87% of the time when considering predictions made throughout the entire hospital stay. However, this accuracy dropped to 62% when utilizing patient data recorded before the patient met sepsis criteria and decreased to 53% when predictions were limited to before a blood culture had been ordered.
"We suspect that some of the health data that the Epic Sepsis Model relies on encodes, perhaps unintentionally, clinician suspicion that the patient has sepsis," Jenna Wiens, PhD, a corresponding author of the study and associate professor of computer science and engineering at the University of Michigan, said in a news release.
Epic told Becker's after the release of the Michigan study that multiple organizations have studied the same sepsis model as was included in University of Michigan's study, but in clinical settings using different workflows.
"They have published on the positive impact that Epic's sepsis model has had on patient outcomes, such as reducing the odds of sepsis mortality by 44% and improving the timeliness of antibiotic administration by about 40 minutes without increasing antibiotic usage," the spokesperson said.