AI can’t distinguish between true and false medical claims once they’re packaged as realistic-looking hospital notes or social-media health discussions, researchers from New York City-based Mount Sinai Health System found.
The study authors analyzed over 1 million prompts across nine leading large language models after exposing them to actual hospital discharge summaries with a single fake recommendation added, widespread health myths from Reddit, and 300 brief clinical scenarios generated and validated by clinicians, according to the study published Feb. 9 in Lancet Digital Health. Several models treated the falsehoods as real medical guidance.
“AI has the potential to be a real help for clinicians and patients, offering faster insights and support,” said co-senior and co-corresponding author Girish Nadkarni, MD, chief AI officer of Mount Sinai Health System, in a Feb. 9 news release. “But it needs built-in safeguards that check medical claims before they are presented as fact. Our study shows where these systems can still pass on false information, and points to ways we can strengthen them before they are embedded in care.”
Becker’s 3rd Annual Fall Chief Pharmacy Officer Summit is the top tier event that the people leading the future of pharmacy convene. This 3-day event is built specifically for Chief Pharmacy Officers and senior pharmacy leaders. The sessions examine the workforce, financial and clinical priorities shaping pharmacy today. Becker’s is where candid discussions flourish and new ideas thrive. Becker's 3rd Annual Fall Chief Pharmacy Officer Summit
