Researchers from Boston-based Beth Israel Deaconess Medical Center conducted a study where they pitted ChatGPT-4 against clinicians. The researchers inputted a set of five example medical cases into the chatbot that had been given to clinicians in a previously published survey on probabilistic reasoning.The researchers then gave ChatGPT an identical prompt 100 times, soliciting the likelihood of a specific diagnosis based on the patient’s presentation.
They also tasked the chatbot with updating its estimates in response to certain test results, such as mammography for breast cancer. The team then compared the probabilistic estimates with responses obtained from the survey, which encompassed more than 550 human practitioners.
The researchers found that in all five cases, ChatGPT-4 demonstrated superior accuracy compared to human clinicians when assessing pretest and post-test probability following a negative test result. The large language model did not perform as well after positive test results, however.
Becker’s 3rd Annual Fall Chief Pharmacy Officer Summit is the top tier event that the people leading the future of pharmacy convene. This 3-day event is built specifically for Chief Pharmacy Officers and senior pharmacy leaders. The sessions examine the workforce, financial and clinical priorities shaping pharmacy today. Becker’s is where candid discussions flourish and new ideas thrive. Becker's 3rd Annual Fall Chief Pharmacy Officer Summit
