Clinicians at the Omaha-based University of Nebraska Medical Center conducting the study found the chatbot’s answers to open-ended questions “frequently redundant and cyclical in nature.”
ChatGPT scored just 26.7 percent on open-ended questions, according to a June 6 Wolters Kluwer news release.
Previously the AI tool faired well on the United States Medical Licensing Exam and at providing empathetic answers to patient questions.
“ChatGPT not only has a low rate of correct answers regarding clinical questions in urologic practice, but also makes certain types of errors that pose a risk of spreading medical misinformation,” said Christopher Deibert, MD, one of the study authors from the University of Nebraska Medical Center.
At the Becker's 11th Annual IT + Revenue Cycle Conference: The Future of AI & Digital Health, taking place September 14–17 in Chicago, healthcare executives and digital leaders from across the country will come together to explore how AI, interoperability, cybersecurity, and revenue cycle innovation are transforming care delivery, strengthening financial performance, and driving the next era of digital health. Apply for complimentary registration now.
