Traditional clinical decision support tools outperform generative AI in diagnosing disease, Mass General Brigham researchers found.
The Somerville, Mass.-based health system has had a diagnostic decision support system called DXplain since 1984.
For a May 29 JAMA Network Open study, researchers compared the technology to large language models such as ChatGPT and Gemini. The investigators fed 36 patient cases into the three systems and found that, with lab data, all three models listed the correct diagnosis most of the time (72% for DXplain, 64% for ChatGPT and 58% for Gemini). Without lab data, only DXplain had the correct diagnosis the majority of the time (56% compared to 42% for ChatGPT and 39% for Gemini), though those results weren’t statistically significant.
“We think combining the powerful explanatory capabilities of existing diagnostic systems with the linguistic capabilities of large language models will enable better automated diagnostic decision support and patient outcomes,” said corresponding author Mitchell Feldman, MD, of Mass General Brigham’s Laboratory of Computer Science, in a May 29 news release.
