Physicians preferred AI-generated discharge summaries over human-written ones for 95% of longer hospital stays, according to a Stony Brook (N.Y.) University study published online Oct. 2 in npj Digital Medicine.
Researchers used OpenAI’s GPT-5.2, accessed through a HIPAA-compliant Microsoft Azure instance, to draft summaries for 60 internal medicine hospitalizations at Stony Brook University Hospital. The stays lasted seven to 21 days and took place in 2023 and 2024. Clinical notes came from the hospital’s Oracle Cerner EHR. The pipeline split each record into 50,000-token segments so it could handle charts too large for the model’s context window.
Six hospitalists and six primary care physicians reviewed the summaries. Each encounter was rated by one hospitalist and one PCP. Reviewers scored the AI summaries higher for overall quality, 4.71 versus 3.16 on a 5-point scale. The AI summaries also scored higher for readability, factuality and completeness.
The AI summaries had fewer physician-flagged errors per summary, 0.42 versus 0.73. Most of that gap came from fewer omissions. There was no significant difference in the estimated severity or likelihood of harm from errors. Reviewers rated 93.5% of the 31 radiology incidental findings flagged by the model as factually correct.
PCPs rated the AI summaries easier to understand, easier to explain to patients and caregivers, and more useful for follow-up care.
“These results provide evidence that artificial intelligence can support discharge documentation for complex hospital stays,” said Rachel Wong, MD, co-corresponding author and clinical associate professor of internal medicine and biomedical informatics at Stony Brook’s Renaissance School of Medicine.
Dr. Wong and her co-authors said physician review remains essential before any AI draft enters a patient record. The study took place at a single center, and reviewers knew which summaries were AI-generated.
Co-corresponding author Fusheng Wang, a professor of biomedical informatics and computer science, said Stony Brook Medicine is considering a pilot of the tool in everyday clinical practice.
Becker’s 3rd Annual Fall Chief Pharmacy Officer Summit is the top tier event that the people leading the future of pharmacy convene. This 3-day event is built specifically for Chief Pharmacy Officers and senior pharmacy leaders. The sessions examine the workforce, financial and clinical priorities shaping pharmacy today. Becker’s is where candid discussions flourish and new ideas thrive. Becker's 3rd Annual Fall Chief Pharmacy Officer Summit
