In a JAMA study released Jan. 2, researchers found that ChatGPT’s use of large language models was not well-suited to giving accurate pediatric diagnoses.
By using the prompt ‘List a differential diagnosis and a final diagnosis’ and providing ChatGPT version 3.5 with a description of a clinical, pediatric issue, researchers were attempting to discover the accuracy of ChatGPT’s clinical knowledge. What they found was a diagnostic error rate of 83%, though some incorrect answers were clinically related to the correct diagnosis.
Though initial studies have not yielded promising results, the team behind the study doesn’t believe that LLMs should be entirely discounted from medicine. However, the study advises the use of chatbots such as ChatGPT in non-clinical settings, citing the fact that clinical experience cannot be replaced.
At the Becker's 11th Annual IT + Revenue Cycle Conference: The Future of AI & Digital Health, taking place September 14–17 in Chicago, healthcare executives and digital leaders from across the country will come together to explore how AI, interoperability, cybersecurity, and revenue cycle innovation are transforming care delivery, strengthening financial performance, and driving the next era of digital health. Apply for complimentary registration now.
