For some health system technology leaders, rolling out AI tools was the easy part. Proving they are still worth paying for is the harder, ongoing job.
Shahidul Mannan, chief data officer at Boston Medical Center, put the sharpest number on that discipline during a panel discussion Sept. 15 at Becker’s 11th Annual IT + Revenue Cycle Conference. Each ambient AI license costs the health system between $200 and $600 per provider, and that price tag does not buy a permanent seat.
“We always take a look at the utilization, and if the utilization is falling, do we need more training, or maybe we can use it better in some other places?” he said.
The same logic extends to the health system’s broader AI governance structure.
“A big part of it is not only compliance — privacy, security, safety — but also the value extraction part of it,” Mr. Mannan said. “Are we having duplication with the EHR? Are there too many pilots going on?”
Kushal Patel, MD, system chief medical information officer at Rockford, Ill., Mercyhealth of Wisconsin and Illinois, said that discipline matters because the gap between a good tool and a great one rarely shows up the way vendors present it.
Mercyhealth has been live with the ambient documentation tool Abridge for more than a year and a half, alongside AI tools for chart summarization and drafted provider messaging, and Dr. Patel said chasing marginal performance differences between competing products is often a waste of energy.
“Best in class is great in theory, but when you translate into day-to-day practice, the adoption and utilization shows you that even though a technology maybe 99% comparing another technology 95%, that gap of 4% doesn’t really translate to be as impactful,” he said.
He traced that view back to his own read of the job itself.
“Clinical informatics is more of a social science at times,” Dr. Patel said, pointing to the work of aligning a tool with real clinical workflows rather than optimizing for a benchmark.
At Rochester, Minn.-based Mayo Clinic, that governance discipline plays out at a scale few health systems have to manage. Edwina Bhaskaran, MSN, RN, chief clinical systems and informatics officer, said her team is running 343 concurrent technology projects, some of them large-scale system transitions happening in parallel with everything else.
Deciding what to keep requires being honest about what is not working, she said.
“We need to be truthful to ourselves in terms of what tools we can actually scale,” Ms. Bhaskaran said. “What do we need to decommission? What have we failed?”
Far from a defeat, she described retiring a tool as one of the more satisfying parts of the job.
“My physician partner and I actually thrive on that ability to decommission a system,” she said. “It’s a sense of pride, as an organization, to be able to say we want to decommission something and remain good stewards of the resources we have.”
Robert Bart, MD, chief medical information officer at Pittsburgh-based UPMC, said all three of those approaches point to the same underlying principle: The technology itself is rarely the differentiator. Even as UPMC completes a yearslong consolidation onto a single Epic instance, he said the platform decision matters less than the discipline applied around it.
“You have to have good platforms,” Dr. Bart said. “You have to have good people, or the right people doing the right process, and then you get them on the right platform. That’s actually when you get things really unbelievable.”
He credited that same people-first approach with UPMC’s reputation for clinical performance even during a long stretch of running nine EHRs at once.
“It is not just about the technology,” Dr. Bart said. “It is the culture and the people that are there.”
None of the leaders framed the aforementioned challenges as a reason to slow adoption. For example, Mr. Mannan said Boston Medical Center is preparing to expand its use of AI voice tools in the next few months specifically because of the access and cost savings involved, not despite the governance overhead.
But each leader made clear that the governance work — tracking utilization, retiring what does not scale, resisting the urge to chase marginal feature gaps — is now as much a part of the AI rollout as the go-live.
