UPMC may slow or hold off on some artificial intelligence deployments if the health system does not have the technology infrastructure needed to adequately monitor the tools, according to its chief medical information officer.
The approach comes as health systems work to build governance capabilities around an increasingly crowded AI environment. More than 90% of health systems have deployed third-party AI solutions, but just 44% have a dedicated platform for testing those tools before rollout, according to an Aug. 6 report from the Center for Connected Medicine at UPMC and KLAS Research.
Rob Bart, MD, chief medical information officer at Pittsburgh-based UPMC, told Becker’s that the health system is working to ensure it has the infrastructure not only to support AI platforms but also to continuously monitor them after implementation.
“In some cases, the answer is yes,” Dr. Bart said. “In other cases, we still need to find an appropriate technology solution, or we have to hold off or slow down some of our deployment until we have those capabilities.”
The challenge is becoming more pressing as the number of AI tools health systems use grows. Dr. Bart said relying solely on people to repeatedly monitor algorithms can quickly run into workforce constraints, particularly given the limited number of data scientists available to health systems. Dedicated technology platforms can help augment those teams and support ongoing evaluation of AI tools, he said.
The issue reflects a broader divide identified in the UPMC-KLAS report. Although 92% of surveyed health systems said they test third-party AI tools before deployment, nearly two-thirds, 63%, characterized their overall AI strategy as developing or ad hoc. Health system leaders cited limited resources, insufficient time and shortages of specialized talent among their leading barriers to adoption.
For UPMC, managing that risk is also influencing which AI use cases the health system prioritizes.
Dr. Bart said UPMC has focused predominantly on tools designed to make physicians, advanced practice providers and other staff more efficient or effective rather than leaning heavily into AI that directly informs clinical diagnosis.
“If we can make them more efficient and effective with those, we can return time to them for the cognitive, the things that are needed in thinking about their patients from a clinical diagnostic, clinical management perspective,” he said.
That strategy also allows UPMC to introduce AI in areas that may carry less clinical risk while both the technology and healthcare’s governance capabilities mature, according to Dr. Bart. He said the health system could move further into clinical diagnostic applications as confidence in their safety grows and regulatory evaluation evolves.
Clinical documentation already represents the most common AI use case among health systems surveyed in the report, cited by 52% of respondents. Revenue cycle, coding and billing applications followed at 36%.
UPMC’s experience with ambient documentation also illustrates why Dr. Bart cautions against evaluating AI solely through traditional financial return on investment.
He said the technology has reduced after-hours documentation — often called “pajama time” — for many UPMC physicians from about two hours per night to less than 30 minutes. UPMC did not adopt the technology on the premise that physicians would necessarily see more patients, he said. Instead, reducing administrative burden and helping address clinician burnout represented value in itself.
As health systems build stronger governance programs, Dr. Bart said the challenge will be balancing those potential benefits against patient safety without creating controls so restrictive that useful technology cannot reach clinicians.
“We always want to hedge on the side of safety, but we don’t want to hedge to the point where we’re withholding some of the benefits that AI could afford clinicians and patients in the care process,” he said. “We need to do it with the right guardrails in place.”
That balance is likely to become more consequential as AI becomes embedded across more healthcare technologies. Dr. Bart said the industry is already reaching a point at which avoiding AI entirely may no longer be realistic.
“It is not possible, in my view, to get care in the United States that is 100% free of artificial intelligence,” he said.
At the Becker's 11th Annual IT + Revenue Cycle Conference: The Future of AI & Digital Health, taking place September 14–17 in Chicago, healthcare executives and digital leaders from across the country will come together to explore how AI, interoperability, cybersecurity, and revenue cycle innovation are transforming care delivery, strengthening financial performance, and driving the next era of digital health. Apply for complimentary registration now.
