Artificial intelligence is moving faster than healthcare organizations can govern it and the executives trying to keep pace know it.
That tension sat at the center of a recent panel discussion at the Becker’s 16th Annual Meeting on AI investment strategy, where health system leaders and industry experts wrestled with a question that has no clear answer: how do you build the infrastructure to manage technology that evolves faster than your ability to evaluate it?
Jeff Gautney, senior vice president and CIO at Rush, a $3.5 billion health system in Chicago, keeps the frontline teams involved for a smooth adoption into existing workflows.
“Whatever we deploy from an AI perspective has got to not break the workflow,” he said. “People are heads down doing their work. If I make them get up and do something else over here and then bring that mentally back to this, I’ve lost a lot of the value that I’ve gotten out of the tool.”
But workflow integration is only the beginning. Once AI is running inside an organization, the harder problem is knowing whether it’s running well. Mr. Gautney said contained models — those that pull data, run a protocol, and produce a consistent output — are easier to monitor than patient-facing tools that learn continuously.
“You’re giving somebody actual advice about the kind of care they might receive and where to receive it,” he said. “That level of monitoring is much different.”
What makes that monitoring so difficult is the fundamental difference between how IT teams are trained to think about systems and how AI actually behaves.
“Normally clinicians expect that everything with IT has a 100% confidence factor. We test it end to end. Every time I put in A, I will always get B,” said Mr. Gautney. “AI is not that.”
The failure mode is gradual and not always obvious, which can be frustrating and risky in the patient care setting.
“[The AI model] goes crazy a little bit at times until it’s full of psychosis,” he said. “You need your front line, the people actually working with the system, to have their spidey sense turned on all the time and start to tell you when things are going wrong.”
Steve Kearney, global medical director at SAS Institute, brought a cross-industry lens to the challenge, noting that other sectors have been stress-testing AI monitoring at scales that dwarf most health systems. In banking, model monitoring happens in 100 milliseconds. In manufacturing, 15,000 models run in real time. The lesson healthcare can draw from that experience is less about the technology itself and more about the conditions required for trust.
“AI moves at the speed of trust,” he said. “If you can’t trust those folks that you’re working with, then it doesn’t matter how good your system is.”
Building that trust starts with getting the math right and being willing to show your work through peer-reviewed publications, validated environments and third-party evaluation. SAS has positioned itself in many organizations as an arbiter of other vendors’ tools.
At Montefiore Health System, a 10-hospital academic medical center in New York, Sudipto Srivastava, chief data and analytics officer, is grappling with a universal structural challenge: healthcare organizations are far better at adopting technology than retiring it.
“We’re really good at addition, we’re really bad at subtraction,” he said. “Suddenly you have an ecosystem of multiple agents running. How are you going to make sure the agents are talking to each other? What’s the orchestration layer that’s doing the quarterbacking of all these things?”
Mr. Srivastava described Montefiore’s governance evolution as a series of rapid iterations driven by necessity. The organization launched governance 1.0 roughly a year ago with a focus on speed — a small decision-making body empowered to move quickly. But speed created its own bottleneck when clinical and ROI-focused stakeholders reached different conclusions. Governance 2.0 brought everyone into the same room. Now the organization is already designing Governance 3.0, built around a tiered risk matrix.
“A large governance structure is built for deliberation, but it is not built for speed,” Mr. Srivastava said. Low-risk decisions get rubber-stamped and move forward. Medium-risk decisions get a lighter review. Only the most complex or expensive proposals require full strategic input.
“It’s okay that you find something that is not working and make something better because the speed at which things are changing is super fast,” he said.
Mr. Gautney’s approach at Rush followed a similar arc. Before a formal AI center of excellence existed, Rush stood up an Office of Responsible AI to audit agreements already signed that contained AI components. From there, the Center for AI Excellence took shape, focused on intake, business case evaluation, literacy and monitoring. Now even that structure is being retooled. The original model, essentially asking people to voluntarily disclose what they’d already deployed, worked well for creating an inventory but it rewarded the loudest voices rather than surfacing the highest-value opportunities.
“What we found was people who had passion around this or had a vendor that was very persistent in their ear were the ones that brought ideas,” he said. “They’re all valid on some level. But that doesn’t necessarily mean we’re working on the things that are directly connected to the strategy.”
Rush has since developed a tiger team of three — a process architect and two forward-deployed engineers — tasked with proactively mapping AI to Rush’s business strategy. Their first focus was revenue cycle, chosen deliberately because of the mounting pressure from payer actions.
“We partnered with our revenue cycle leader, broke down all of the threats they’re currently facing, and methodically went through the process and identified AI solutions, starting with Epic, staying core to the pillar, but adding on point solutions where Epic wasn’t there yet,” said Mr. Gautney. “That’s been our most successful deployment.”
As technology evolves, CIOs are seeing a new challenge emerge. Companies are adding AI tools through updates instead of the formal procurement process. Tools that organizations have been using for years — Workday, DocuSign, Epic — are embedding AI into their core functionality without the option for oversight; it just turns on.
“One day DocuSign could summarize all my contracts for me. It’s super useful, but we never talked about it. It just showed up as part of an upgrade,” Mr. Gautney said.
Some of those enhancements carry additional costs that surface only after adoption. “You need a FinOps model for it,” he said.
“I think 18 months or less from now, it’s just going to be the way every solution we own is implemented and works,” Mr. Gautney said. “We have to not have a separate group that just deals with AI. Everybody in IT has to be able to deal with AI because it’ll be in every single thing we do.”
For the C-suite, the shift has been liberating and destabilizing. Rush rolled out Copilot to roughly 3,000 knowledge workers and gave executives the ability to query their own data directly — without waiting for an analyst to pull a report or reconcile conflicting dashboards.
“They’re able to ask questions that in the past they had to look at a couple of different dashboards or ask for an extract,” Mr. Gautney said. “By curating the data and giving them the tools to go ask the questions themselves and drill down, that’s very positive in terms of their ability to pivot strategy and react to things more in real time.”
Rush is now moving that capability into Claude. But the same accessibility that empowers executives has also inflated expectations. “There’s an awful lot of us bringing back into focus what is real, particularly on the green money side,” he said.
Mr. Srivastava saw the C-suite shift differently — as a closing of the translation gap that has long frustrated clinical and operational leaders when they engage with technology teams.
“They would go to the tech teams and they would get some gobbledygook and they’d be like, I don’t know what they’re talking about. But now with agents and other things, they can start imagining — because they have a concept of productivity, how to extract that, how to minimize the silly kind of work.”
At the Becker's 11th Annual IT + Revenue Cycle Conference: The Future of AI & Digital Health, taking place September 14–17 in Chicago, healthcare executives and digital leaders from across the country will come together to explore how AI, interoperability, cybersecurity, and revenue cycle innovation are transforming care delivery, strengthening financial performance, and driving the next era of digital health. Apply for complimentary registration now.
