Clinicians across the country are using AI tools their organizations never approved, often to solve real, urgent problems. The phenomenon, known as shadow AI, sits at the center of a governance challenge that is growing faster than most health systems can keep pace with.
During a featured session, ‘Shadow AI: Governing the unseen with evidence and expertise’ at Becker’s 16th Annual Meeting, hosted by Wolters Kluwer Health, four health system chief medical informatics officers discussed how they are confronting shadow AI — understanding why it happens, quantifying the risks it carries and building governance frameworks designed to channel the behavior rather than simply prohibit it.
The session was moderated by Amanda Heidemann, MD, FAAFP, FAMIA, physician advisor at Wolters Kluwer Health. The panelists were:
- Ann Cappellari, MD, CMIO, SSM Health (St. Louis)
- Bonny Chen, MD, CMIO, Froedtert ThedaCare Health (Menomonee Falls, Wis.)
- Rob Bart, MD, CMIO, UPMC (Pittsburgh)
- Louis Jeansonne, MD, CMIO, Ochsner Health (New Orleans)
Below are four takeaways from their conversation.
Note: Quotes have been edited lightly for length and clarity.
1. Shadow AI signals unmet need — not just unauthorized behavior
The panelists agreed that when clinicians reach for unsanctioned tools, they are usually solving a real problem their organization has not yet addressed. Dr. Jeansonne described the early wave of note-taking bots in virtual meetings as an inflection point: “You can’t just tell people they’re not allowed to use it, because it’s making you aware of a need that people have and it’s something that’s making their lives easier.”
Dr. Cappellari described a more persistent version of the same dynamic at SSM Health, where only about 2% of physicians currently have access to an approved ambient documentation tool despite hundreds on a waiting list. When clinicians download an unapproved app, she said, it becomes difficult to tell them not to — because the approved alternative simply isn’t available to them yet. The implication for health system leaders is that shadow AI is better understood as a diagnostic signal than a compliance failure.
2. The risks run deeper than data privacy
Patient privacy is the most obvious concern when employees enter clinical information into unsanctioned tools, and Dr. Jeansonne noted that even seemingly anonymized inputs can become identifiable when combined with other data. But the panelists identified subtler, harder-to-detect risks as well.
Dr. Bart described an AI scheduling tool his team evaluated that appeared to improve patient access — until a closer review revealed it disadvantaged patients who relied on public transit. The algorithm deprioritized them after missed appointments, compounding access barriers rather than reducing them. That kind of equity risk, he said, would not have surfaced under UPMC’s first-generation AI governance model.
“I think the breadth of how you need to examine something that’s coming in — because we’re not at a mature state anywhere in our healthcare industry — requires that you really dive in deep,” Dr. Bart said.
Dr. Cappellari raised an additional concern: a less experienced workforce increasingly reliant on AI outputs may lack the clinical knowledge to critically evaluate it. If new clinicians don’t know enough to recognize when the tool is leading them astray, she noted, they won’t know to question it.
3. Governance has to be multidisciplinary and built for speed
Dr. Chen outlined a framework she described as the “three E’s”: enable, educate and evaluate. At Froedtert ThedaCare, that has meant inventorying and stratifying every AI tool by risk, publishing the approved list organization-wide and building a governance committee that spans clinical leadership, operations, IT, legal and compliance. The goal is to make the trusted path the easy path.
But even well-built governance structures are being outrun by the pace of AI adoption. Dr. Bart estimated that UPMC could have more than 5,000 AI tools in use by 2030, far exceeding the number of traditional software applications in its environment. “We’re actually examining AI-based platforms to manage and monitor the AI that we’re putting in place because we do not have enough humans, enough qualified data scientists, enough people with the expertise to keep managing and monitoring,” he said.
4. Adoption is the upside of shadow AI
The session closed with a counterintuitive point. Dr. Jeansonne argued that shadow AI, for all its risks, solves one of the most persistent problems in health IT: getting clinicians to actually use new tools. When people are already using an unsanctioned solution, the adoption work is largely done — organizations simply need to offer something equally effective within a trusted framework. “All you have to do is provide something that’s as good, and you know that they will use it,” Dr. Jeansonne said.
Dr. Bart offered a framing he said he repeats to clinicians regularly: “The AI-enabled clinician will be able to out-compete and outperform the AI-naive clinician every day.” The goal of governance, the panelists agreed, is not to shut that drive down; it is to give it a safe place to run.
Where leaders go from here
Shadow AI is not going away, and the panelists were clear that trying to block it entirely is both impractical and counterproductive. The health systems making progress are those that treat unsanctioned tool use as a window into what their workforce actually needs, build governance structures broad enough to catch equity and safety risks that pure IT reviews would miss, and invest in enabling approved tools widely enough that clinicians don’t have to go looking elsewhere. As Dr. Cappellari put it, the job is to fill the gap — not block it.
