Before health systems can meaningfully benefit from artificial intelligence, they have to candidly reflect on age-old questions: Do we actually know what data we have? Where does our data live? Can our data even be trusted?
For a growing number of CIOs and IT executives, these questions are getting real. They are the gating factor for AI adoption, and require data cleanliness. Organizations that skip this step aren’t just setting themselves up for failure; they’re emphasizing failure and increasing regulatory exposure.
“Everyone’s racing to pick the right model or the right vendor, but the organizations that treat this as a procurement decision are going to be deeply disappointed, and in some cases, they’re going to create serious risk in the process,” Matt Morton, assistant vice president and chief information security officer for the University of Chicago, told Becker’s. “AI will be transformative in healthcare, but it’s only as good as what you fed it and in an environment already burdened by fragmented data, legacy infrastructure and inconsistent data practices, deploying AI without solid data governance isn’t innovation; it’s amplification of the problem.”
Health systems focused on deploying AI too quickly risk operational inefficiencies and regulatory risk. The policies around data governance are evolving and staying compliant with HIPAA is a top concern. IT leaders must also guard against AI models inappropriately influencing data and output.
“Without knowing what data you have, where it lives and how it’s classified, and who has access to it, you’re not just risking poor model performance – you’re creating HIPAA exposure and potentially introducing bias into clinical workflows,” said Mr. Morton.
Eric Snyder, executive director of technology and innovation for the Wilmot Cancer Institute at the University of Rochester (N.Y.) Medical Center, said the data problem is a fundamental sequencing issue. Health systems are reaching for AI before they’ve done the foundational work to make it trustworthy.
“Models are easy to get; what’s hard is clean, trustworthy data, governance and integration into real clinical and operational workflows,” he said. “Without curation, validation and context, AI just scales bad assumptions faster.”
Nobody wants that. Many organizations have spent the last year-plus refining governance structures for AI to make initial purchasing decisions as well as monitor progress over time. Some health systems have developed AI governance committees and new guideposts specifically designed for the quickly evolving technology while others have elevated existing governance infrastructure.
But Andrew Rosenberg, MD, CIO of Michigan Medicine in Ann Arbor, observed that health systems believing they have an AI-ready governance infrastructure are often operating with false confidence. He sees leaders unaware of how nascent their actual maturity is in areas like AI observability.
“The more nuanced issues, such as AI observability categories – data quality, model performance and drift, infrastructure monitoring and user feedback – are nascent at best,” he said. “We have long way to go if various AIs will be used in the same manner we have become accustomed and assured as we use medical devices, pharmaceuticals, laboratory practices and behavior of licensed clinicians.”
The downstream consequences of data governance failures isn’t always a system crash or regulatory citation. Sometimes it’s just predictions that look accurate but don’t translate into clinical action because nobody owns the alert. Teams may also find themselves in workflows without an explicit accountability structure, leading to daily breakdowns and poor outcomes.
Chantal Fremont, DNP, corporate director of nursing performance standards and innovation for Emory Healthcare in Atlanta, has seen the gap between prediction and execution firsthand.
“People assume better prediction automatically equals better outcomes when prediction without execution is just a dashboard,” she said. “Real impact happens when the alert has a clearly assigned owner, the accountability structure is explicit, workflow is frictionless and the frontline trusts them too. AI doesn’t transform healthcare; systems that know how to operate intelligence do.”
Health systems that will benefit from AI the most are not the ones that deploy it the fastest. Instead, it will be the systems investing in the foundational infrastructure for responsible deployment.
“Without strong data governance, workflow integration and clinical oversight, AI adds risk rather than value,” said Babatope Fatuyi, MD, CMIO of UTHealth Houston. “Healthcare organizations that treat AI as a tool for resilience and efficiency, rather than a shortcut, see the most meaningful results.”
At the Becker's 11th Annual IT + Revenue Cycle Conference: The Future of AI & Digital Health, taking place September 14–17 in Chicago, healthcare executives and digital leaders from across the country will come together to explore how AI, interoperability, cybersecurity, and revenue cycle innovation are transforming care delivery, strengthening financial performance, and driving the next era of digital health. Apply for complimentary registration now.