Health system digital leaders told Becker's they use different generative artificial intelligence programs depending on the situation, but "Dr. ChatGPT" hasn't arrived just yet.
As large language models including ChatGPT, Google Bard and Microsoft Bing AI have grabbed the world's attention, healthcare executives have been figuring out how they can streamline administrative functions and how they might eventually help patient care.
"I've tried all of them, including Glass Health," said Anthony Chang, MD, a pediatric cardiologist and chief intelligence and innovation officer at Orange, Calif.-based Children's Hospital of Orange County. "Glass is really best in terms of a clinician looking for a differential diagnosis in the clinic setting. But ChatGPT has been good for logistical issues like writing a justification letter for a patient for a procedure. But when it comes to a difficult case in the clinic, Glass Health has been the best by far."
He said he subscribes to ChatGPT's paid version, which employs its GPT-4 update (the free version is GPT-3.5). He said the update is more detailed and writes better. He said he "wasn't impressed" by Google and Microsoft's offerings.
For generative AI to reach its true promise in healthcare, however, it will have to add "weight" to its words and not just give a simple thumbs up or down. "Someone who's had diabetes for 10 years, that's much weightier than someone who had a headache yesterday and it's gone today," he explained. "We're definitely not ready for Dr. ChatGPT."
Michael Hasselberg, PhD, RN, chief digital health officer of University of Rochester (N.Y.) Medical Center, said large language models are trained on general-knowledge data, so he's found they perform well on more general healthcare questions but not specialized queries.
"We feel that the important distinction is not between which of these large models is the best choice for healthcare-related questions but the distinction between when we should rely on these large models and when we should use smaller models specifically trained on healthcare data," he said. "To reach their full potential and ensure the trust of the healthcare community, all of these models will need continuous improvements in their understanding of medical context and have ongoing monitoring of their accuracy through clinical expert validation."
B.J. Moore, CIO of Renton, Wash.-based Providence, agreed that it's "too early to name a winner in the race for generative AI in healthcare."
However, the health system is focused on Microsoft because of its partnership with the tech giant, he said. Microsoft, which uses technology from ChatGPT developer OpenAI, is also incorporating generative AI into clinical documentation through its Nuance subsidiary and patient portal messages via a collaboration with Epic.
"There are a multitude of use cases for this technology across healthcare — patient-facing, clinician-facing, and administrative — and we may find over time that each chatbot family will prove stronger in different areas," Mr. Moore said. "This will be a function of their design and the datasets they are trained on."
Providence is focusing on administrative and provider productivity uses now and plans to explore clinical use cases in the future, he said. To get better, the programs will have to filter out "hallucinations" and trace back their conclusions to data.
"AI needs to be a co-pilot or assistant first," he said. "Caregivers must be at the center of the process. As they gain trust and confidence in AI, they will drive adoption in administrative processes and ultimately at the point of care."
Deepesh Chandra, chief analytics officer of Cincinnati-based Bon Secours Mercy Health, said none of the apps are "universally applicable" for healthcare. He said generative AI still needs to solve for regulatory compliance, drawing from trusted source material, transparency and explainability, and human oversight and accountability.
"Given the sensitivity of healthcare, delivering tangible value requires a deep understanding of specific workflows, clinical guidelines and risks," he said. "While popular generative AI tools present massive upside, they all come with significant risks around false information and bias — which does not align with our trust-based relationships with healthcare providers."