In a study published April 19 in NEJM AI, the researchers gathered more than 27,000 distinct diagnosis and procedure codes from a year of routine care at Mount Sinai Health System. They then utilized descriptions for each code to trigger models from OpenAI, Google and Meta to produce the most precise medical codes.
The codes generated were then compared with the original ones, and any patterns in errors were examined, according to an April 22 news release from Mount Sinai.
All of the large language models analyzed — GPT-4, GPT-3.5, Gemini-pro and Llama-2-70b — demonstrated accuracy below 50% in replicating the original medical codes, the researchers found.
GPT-4 showed the best performance among the models. It achieved the highest exact match rates for ICD-9-CM (45.9%), ICD-10-CM (33.9%) and CPT codes (49.8%).
Becker’s 3rd Annual Fall Chief Pharmacy Officer Summit is the top tier event that the people leading the future of pharmacy convene. This 3-day event is built specifically for Chief Pharmacy Officers and senior pharmacy leaders. The sessions examine the workforce, financial and clinical priorities shaping pharmacy today. Becker’s is where candid discussions flourish and new ideas thrive. Becker's 3rd Annual Fall Chief Pharmacy Officer Summit
