Two years ago, Falls Church, Va.-based Inova Health System was reviewing roughly one artificial intelligence feature for approval each month. Today, that number has climbed to more than a dozen, prompting the health system to rethink how it evaluates and manages an expanding pipeline of AI projects.
Inova now has 92 AI features in production and is preparing its governance infrastructure to oversee as many as 500, Jon McManus, vice president and chief data and AI officer at Inova, told Becker’s.
The growth has been rapid. Just one year ago, Inova was reviewing approximately three or four AI features each month. Now, the health system evaluates more than 12 monthly for approval, with proposals spanning new capabilities from existing technology vendors, ideas generated by employees and initiatives emerging from partnerships with companies such as OpenAI and Anthropic.
The challenge is no longer simply identifying promising AI applications. It is managing an expanding pipeline without allowing oversight processes to slow implementation.
“We centralize governance, and we centralize post-deployment monitor,” Mr. McManus said. “What’s important is we don’t create an unnecessary bottleneck to the implementation of AI.”
Rather than having one centralized team responsible for implementing every AI initiative, Inova distributes implementation responsibilities across the organization while maintaining centralized governance and post-deployment monitoring.
That structure allows the health system to work on more than 30 AI projects simultaneously, according to Mr. McManus.
Before an AI feature can move forward, Inova evaluates it against four pillars: value, safety, reliability and impact. The health system also considers whether a proposed application aligns with organizational priorities, including growth, workforce well-being and cost resilience.
Even with those criteria, the volume of potential projects can exceed what the organization has capacity to pursue.
To manage that demand, Inova has established an AI governance platform designed to support evaluations and oversight at scale.
“I think it’s important that health systems graduate from spreadsheets,” Mr. McManus said.
The platform uses AI agents to assist with market intelligence and preparation of governance documentation, while keeping humans involved in the review process.
Inova has also developed specialized roles to support its expanding AI portfolio.
A team of AI product managers focuses on vendor oversight, administration, risk and governance. Separately, the health system has established an AI solution design team to help departments translate their ideas into technical requirements before projects reach engineering teams.
The distinction matters because not every AI project follows the same path.
When an idea emerges, Inova first determines whether it meets the organization’s criteria for an AI solution. From there, leaders decide whether the technology should come from an existing vendor or be developed internally.
Vendor-provided solutions move through the AI product management and governance processes. Internally developed applications receive additional design support to identify appropriate platforms and technical requirements.
For example, Mr. McManus said Inova favors Epic’s agent platform for initiatives involving changes within the EHR. Other projects may use platforms from OpenAI, Anthropic or additional technology partners, depending on their requirements.
But Mr. McManus said technology and governance structures alone are not enough to manage AI adoption at scale.
“AI literacy is the key differentiator to manage and wield AI decision making at scale effectively,” he said. “I may be the company’s AI officer, but I need my entire company making good decisions around AI to be successful.”
Inova has developed AI education programs for employees, clinical teams and organizational leaders. Its leadership curriculum includes helping executives identify problems suited for AI and understand how multiple AI applications can work together to address broader operational challenges.
The goal is to equip leaders to make informed decisions about AI using the organization’s existing investment and resource allocation processes rather than establishing entirely separate structures for every AI initiative.
Inova is also confronting another challenge that comes with a growing AI portfolio: determining whether deployed technologies continue to meet performance expectations.
The health system conducts quarterly safety and reliability assessments of its production AI features, collecting feedback from operational owners and combining those evaluations with quantitative performance measures.
Its most recent quarterly review identified three AI features whose performance scores had declined enough to warrant additional committee scrutiny. Inova subsequently developed decommissioning plans for all three.
“I think that’s a big milestone that organizations aren’t just turning AI on; they are purposely turning AI off if it is not meeting performance standards,” Mr. McManus said.
As Inova prepares to manage an increasingly large AI portfolio, Mr. McManus said other health systems should focus on strengthening their workforce’s AI capabilities and reinforcing existing decision-making processes before creating entirely new structures dedicated to the technology.
“AI will not be novel forever,” he said.
Becker’s 3rd Annual Fall Chief Pharmacy Officer Summit is the top tier event that the people leading the future of pharmacy convene. This 3-day event is built specifically for Chief Pharmacy Officers and senior pharmacy leaders. The sessions examine the workforce, financial and clinical priorities shaping pharmacy today. Becker’s is where candid discussions flourish and new ideas thrive. Becker's 3rd Annual Fall Chief Pharmacy Officer Summit
