For years, the conversation about AI in health systems centered on technology adoption: which tools to buy, which pilots to run, which workflows to automate. But as health systems move from isolated AI deployments toward enterprise-wide agentic platforms, the limiting factor is no longer the technology. It’s the people managing it.
That was the central tension running through a panel of health system technology leaders at Becker’s 16th Annual Meeting in Chicago this spring. Across organizations ranging from a large rural integrated delivery network to an urban academic medical center to a national cancer system, the same challenge surfaced: operations leaders have not yet grasped that they are now managing a digital workforce — and the consequences of that gap are starting to show.
“The biggest barrier to us moving forward is really getting operations to understand that this fundamentally changes their role in the equation,” said Jeff Gautney, CIO of Rush University Medical Center in Chicago. “They are managing a digital workforce and they need to think that way as opposed to [thinking that] IT is monitoring this, IT is keeping an eye on it, IT is delivering this solution and I don’t really need to think any differently about it.”
Rush is roughly 18 months into its agentic AI journey, which began with an early adoption of Salesforce AgentForce. Mr. Gautney was candid about initial challenges with the technology as it evolved, but in partnership with Salesforce, Rush was able to get back on track. The health system learned that agentic AI doesn’t fail the same way as traditional software; there isn’t one clean break. Instead, they found agents can slowly drift and the people involved need to monitor for small issues and course correct quickly.
“An interface breaks, it’s binary, it’s on or off,” said Mr. Gautney. “[AI agents] go slowly crazy usually, right? And IT can recognize when it’s going wildly out of whack. But when it’s just starting down that path, that’s got to be operations, and that is a fundamental change in how they think about their jobs. That is what the next 18 or 24 months is going to be, turning on the lights for operational leaders.”
That distinction — between binary failure and gradual degradation — carries significant implications for how health systems assign accountability for AI performance. At Rush, it has accelerated the push to reframe what operations oversight means in an environment where AI agents run continuously, consume resources constantly, and can slowly deviate from their intended behavior without any obvious alarm.
Mr. Gautney also mentioned a related financial risk that health systems are beginning to encounter. Unlike traditional software licenses, agentic AI runs on token consumption, meaning costs accrue around the clock regardless of utilization patterns.
“When you turn on an agent, it’s on all the time,” he said. “It is consuming tokens all the time, and your pricing model very quickly — it’s easy to see how the agent becomes more expensive to operate than the people that it’s replacing.”
The solution, he argued, requires a financial operations discipline most health systems have not yet developed for AI.
One of the more instructive stories from the panel involved something as operationally mundane as clinic hours. Rush deployed an agentic tool to surface accurate schedule information and discovered that the clinic’s process for updating its own hours involved an email to marketing, which then emailed a website vendor, creating a lag of two to three days. The agent simply grabbed whatever was in the system.
“What looked like a hallucination was actually bad data and bad process behind the bad data,” Mr. Gautney said.
The clinic team had to fix the process before the agent could accurately do its job.
“Agentic AI immediately exposes all of the bad data in your organization and all of the bad process and all of the non-standardized things that you do because it just eats that stuff as fuel,” said Mr. Gautney. “If you have bad fuel, the engine stops working or goes into directions you really don’t want it to go.”
As health systems prepare for deploying a “cluster of agents” — solutions from Epic, Workday, Salesforce, UiPath, and others all pushing agentic tools simultaneously — a new infrastructure priority is emerging: identity management.
“How do we make sure that that agent truly has the rights and is provisioned to do what it wants to do and to ask for and use the data that it wants to use?” Mr. Gautney asked. “If you haven’t invested in identity management, that’s going to be the next essential infrastructure element to make sure you can come through this cloudy period.”
At the Becker's 11th Annual IT + Revenue Cycle Conference: The Future of AI & Digital Health, taking place September 14–17 in Chicago, healthcare executives and digital leaders from across the country will come together to explore how AI, interoperability, cybersecurity, and revenue cycle innovation are transforming care delivery, strengthening financial performance, and driving the next era of digital health. Apply for complimentary registration now.