The question healthcare CIOs are asking about artificial intelligence has changed. A year ago, the debate centered on whether to adopt. Now, it centers on whether organizations can actually see what their AI is doing once it’s in the building.
“One of the most dangerous trends in health IT today is the invisible integration of AI into clinical workflows,” Charity Dorazio, CIO of Adventist HealthCare in Gaithersburg, Md., said. “This can occur when vendors enable new AI features without informing customers or when staff adopt AI tools independently. AI that is not properly evaluated, monitored, and governed can create significant risks, including data exposure, gaps in oversight, and the inability to implement effective monitoring and control processes.”
Ms. Dorazio’s concern reflects a dynamic that has become increasingly common as vendor-embedded AI proliferates: organizations that believe they have governed AI adoption discover they have governed only the tools they know about. Shadow adoption — staff using consumer AI products, vendors silently enabling features — is expanding the perimeter in ways most current governance frameworks were not designed to address.
Carter Smith, managing director of Cone Health Ventures in Greensboro, N.C., has seen the downstream consequence of that visibility gap.
“Most health systems can tell you how many AI tools they’ve launched. Far fewer can tell you which ones are still performing the way they did on day one,” he said. “Models drift, patient populations change, documentation patterns shift, but the monitoring to catch that rarely ships with the pilot.”
The compounding effect is automation bias — the tendency for clinicians and staff to stop questioning a tool once it is embedded in a workflow.
“That’s how quiet degradation in model performance becomes a patient safety or compliance problem nobody sees coming,” he said. His prescription is structural: every AI deployment should have a named owner, a performance baseline and a kill switch. “If a tool can’t clear that bar, it isn’t ready for production.”
Bob Berbeco, CIO of Mahaska Health in Oskaloosa, Iowa, said there is also risk in building AI on top of existing structures if they haven’t been updated.
“The most dangerous trend in healthcare IT is the rush to layer more technology, especially AI, into workflows and data foundations that are not ready for it,” he said. “Organizations that chase innovation without first addressing data quality, interoperability, cybersecurity, governance, and workflow design don’t create value faster. They create complexity and additional risk at an accelerated rate.”
The credibility problem that follows is the deeper concern.
“AI makes this risk especially acute,” he said. “Once it’s embedded in a trusted clinical workflow, people may give its recommendations more credibility than earned. That’s what concerns me — not whether healthcare adopts AI fast enough, but whether we hold onto human judgment, accountability, and trust as we do it.”
Nonku Kunene Adumetey, director of population health data analytics at ChristianaCare Health System in Newark, Del., put the same concern in operational terms.
“When organizations implement tools without validating the data, monitoring for bias or integrating insights into clinical and operational workflows, they risk scaling inaccurate decisions rather than improving care,” she said. “The real opportunity is not simply to adopt more technology, but to ensure it produces trusted, actionable insights that improve outcomes, reduce disparities and support the people delivering care.”
Ray Lowe, senior vice president and CIO of AltaMed in Los Angeles, connected the data governance argument to the cybersecurity surface that AI simultaneously expands.
“AI is only as good as the data behind it, and without strong data stewardship, organizations risk automating bad decisions at scale,” he said. “Every AI investment should begin with a defined business case, measurable KPIs, controlled pilots, and ongoing monitoring of tokenization and consumption costs to ensure value is realized and sustainable.”
Mr. Lowe also flagged a threat vector that governance conversations frequently underweight.
“Cybercriminals are now leveraging AI to accelerate phishing, identify vulnerabilities, and automate attacks faster than organizations can respond,” he said. “The risk is no longer limited to data breaches. A successful cyberattack can disrupt patient care, compromise operations, impact revenue, and erode patient trust.”
Susan Goodson, senior vice president and chief digital information officer at Lurie Children’s in Chicago, argued that the proliferation of AI tools has introduced a different kind of organizational risk: the distraction of chasing capability at the expense of operational foundation.
“The most dangerous trend in health IT is taking our eye off the ball,” Ms. Goodson said. “I see cyber risk rising, AI accelerating expectations, and tighter margins leaving little room for wasted effort. There is so much temptation to chase every new capability or respond to every urgent demand. The most successful organizations are focused on the fundamentals: resilient operations, secure and trustworthy data, strong governance, and deploying technology that has impact for our patients and our employees.”
Not every risk identified sits inside the health system’s own four walls. Judd Hollander, MD, senior vice president for health delivery innovation and chief virtual care officer at Jefferson Health in Philadelphia, pointed to a systemic dynamic that efficient AI on both sides of the claims process could make worse.
“The AI arms race is the most dangerous trend,” Dr. Hollander said. “I fear that health systems will race to utilize AI to decrease denials and improve pre-authorization while payers will race to utilize AI to maintain denials and limit pre-authorization. Thus, we will all drive up healthcare costs reaching the exact same status quo we currently have with respect to denials and pre-authorization.”
The concern is not that AI will perform the function poorly — it is that it will perform it very well, producing a more efficient version of a system whose underlying design is still broken. “We cannot simply make poorly designed systems more efficient,” Dr. Hollander said. “We must re-design the system.”
Across these perspectives, the through-line is less about caution toward AI than about the institutional discipline required to use it responsibly. The leaders who voiced concern are not arguing for slower adoption. They are arguing that the accountability structures — governance frameworks, performance monitoring, data foundations, named owners, kill switches — need to ship with the technology, not follow it. As Mr. Smith put it: “The answer isn’t to slow down.”
At the Becker's 11th Annual IT + Revenue Cycle Conference: The Future of AI & Digital Health, taking place September 14–17 in Chicago, healthcare executives and digital leaders from across the country will come together to explore how AI, interoperability, cybersecurity, and revenue cycle innovation are transforming care delivery, strengthening financial performance, and driving the next era of digital health. Apply for complimentary registration now.
