Healthcare has entered a new phase of artificial intelligence adoption. The question is no longer whether AI can improve healthcare. Predictive models, generative AI, ambient documentation, and intelligent automation have demonstrated their potential across clinical and operational settings.
The real challenge facing health systems today is different. As health systems move from AI experimentation to enterprise deployment, success is becoming less about building better models and more about redesigning how work gets done.
Why do so many promising AI initiatives fail to deliver enterprise-wide impact?
Across the industry, organizations continue to invest millions of dollars in AI. They build increasingly sophisticated models, pilot innovative use cases, and report impressive technical performance. Yet many of these initiatives never progress beyond limited deployments or isolated successes.
The problem is rarely the model itself.
After leading AI initiatives across both payer and provider organizations, I’ve come to believe that the greatest determinant of AI success is not algorithm performance — it is workflow integration. AI creates value only when it helps people make better decisions, reduces cognitive burden, and fits naturally into the way clinicians and operational teams already work.
Healthcare leaders often ask, “How accurate is the model?”
The more important question is, “How will this improve the way our people work?”
That shift in thinking fundamentally changes how organizations approach AI. Instead of optimizing only for model performance, successful organizations optimize for adoption, usability, trust, and measurable business outcomes.
In the coming years, the organizations that realize the greatest return on AI investments will not necessarily be those with the most advanced algorithms. They will be the ones that successfully redesign workflows around human-AI collaboration.
AI does not create value on its own
One of the most common misconceptions in healthcare AI is equating technical performance with organizational success.
Data science teams naturally focus on metrics such as AUROC, precision, recall, calibration, and F1 score. These measures are essential for evaluating model quality, but they do not measure whether AI is improving care delivery or operational performance.
A highly accurate model that clinicians ignore has little value. Likewise, an operational model that disrupts existing workflows or requires additional clicks may never gain meaningful adoption, regardless of its predictive accuracy.
Conversely, organizations often achieve substantial business value from models with only modest improvements in predictive performance when those models are seamlessly integrated into daily operations.
Consider a hospital-at-home program. The objective is not simply to identify patients who may qualify for care at home. The real value comes from helping clinical teams identify appropriate patients more quickly, reducing the number of charts requiring manual review, and enabling clinicians to spend more time delivering care rather than searching for information.
The lesson extends well beyond a single use case.
AI creates value not by replacing human expertise, but by helping experts focus their attention where it matters most.
Different organizations, common challenges
Healthcare is often described as a single industry, but payers and providers operate under very different business models.
Providers are focused on delivering safe, high-quality patient care while improving access, experience, and operational efficiency. Payers focus on managing risk, controlling costs, ensuring regulatory compliance, and improving member outcomes.
Despite these differences, both organizations face the same fundamental AI challenge: their experts are overwhelmed by information.
Clinicians spend valuable time reviewing lengthy patient histories before making treatment decisions. Revenue cycle teams sift through thousands of denials to identify root causes. Care managers monitor large patient populations to determine which patients need immediate attention. Operational leaders navigate multiple dashboards before making critical business decisions.
The problem is no longer access to data.
Healthcare organizations have more data than ever before.
The challenge is helping people find the right information at the right time so they can make better decisions.
This is where AI creates its greatest value—not by replacing expertise, but by reducing cognitive burden and helping experts focus on the work that matters most.
Organizations that recognize AI as a tool for augmenting human decision-making rather than automating it entirely are far more likely to achieve sustained adoption.
Prediction alone is not enough
For more than a decade, machine learning has delivered meaningful improvements across healthcare.
Predictive models identify patients at risk of readmission, prioritize claims for investigation, detect fraud and abuse, optimize staffing, forecast demand, and support countless operational decisions.
These capabilities are valuable.
But prediction alone rarely changes outcomes.
The organizations seeing the greatest return on AI investments are those that successfully connect predictions to operational workflows.
Consider a hospital-at-home program.
A predictive model may accurately identify patients who could safely receive acute care at home. Yet if clinicians still need to manually review hundreds of charts, search multiple systems for supporting information, or complete additional documentation before acting on the recommendation, much of the model’s value is lost.
Now imagine the same prediction integrated directly into the clinician’s workflow. Eligible patients are prioritized automatically, key clinical indicators are summarized, and recommendations are presented within the systems clinicians already use.
The predictive model hasn’t changed.
The workflow has.
That distinction often determines whether AI becomes another dashboard—or an indispensable part of clinical operations.
Healthcare organizations should therefore evaluate AI not simply by asking, “Did the model make an accurate prediction?” but also, “Did it enable a better decision and a better workflow?”
Generative AI expands the opportunity
Generative AI represents the next major evolution in healthcare AI—not because it replaces predictive models, but because it addresses an entirely different challenge.
Healthcare is fundamentally a knowledge-intensive industry.
Every day, clinicians, nurses, administrators, analysts, and executives spend countless hours documenting encounters, reviewing patient records, searching policies, answering operational questions, and synthesizing information from multiple sources.
These activities consume significant time and contribute to administrative burden across the enterprise.
Large language models offer an opportunity to change that.
Rather than predicting which patient is at highest risk, generative AI helps clinicians quickly understand years of longitudinal patient history. Instead of requiring analysts to build repetitive reports, conversational AI allows leaders to interact with enterprise data using natural language. Knowledge retrieval systems can make institutional policies, clinical guidelines, and operational procedures accessible in seconds rather than minutes.
The opportunity extends far beyond clinical care.
Revenue cycle operations, finance, supply chain, human resources, compliance, and executive leadership all depend on timely access to accurate information.
Generative AI can democratize organizational knowledge by delivering the right information to the right person at the right moment.
As organizations explore these capabilities, success will depend less on the novelty of the technology and more on thoughtful governance, workflow integration, and clear measures of value.
Looking ahead, AI agents have the potential to extend these capabilities even further by orchestrating tasks across multiple systems, automating routine workflows, and coordinating information retrieval with minimal human intervention. While still an emerging capability, AI agents reinforce the same principle: their value will ultimately depend not on autonomous decision-making, but on how effectively they augment existing clinical and operational workflows.
Leadership determines success
Technology alone has never transformed healthcare.
People do.
The organizations successfully scaling AI share several characteristics.
First, they treat AI as a strategic capability rather than an isolated technology initiative. Executive sponsorship ensures AI investments align with organizational priorities and measurable business outcomes.
Second, they build multidisciplinary teams from the beginning. Successful AI initiatives bring together clinicians, operational leaders, engineers, data scientists, informaticists, compliance professionals, legal teams, and technology leaders. Each group contributes a unique perspective that strengthens both the solution and its adoption.
Third, they recognize that deployment is only the beginning. They continuously monitor adoption, evaluate workflow impact, collect user feedback, measure business outcomes, and refine solutions over time.
Sustaining value also requires disciplined operational practices—including MLOps, continuous monitoring, model governance, and feedback loops—to ensure AI systems remain accurate, reliable, and aligned with changing clinical and operational environments.
Many AI projects fail not because the technology is inadequate, but because organizations underestimate the effort required to redesign workflows, build trust, and manage change.
The most successful AI leaders spend as much time communicating with stakeholders and improving operational processes as they do discuss models and algorithms.
Enterprise AI is ultimately an organizational transformation initiative—not simply a technology deployment.
The metrics that matter
One of the biggest mistakes healthcare organizations make is measuring AI success the same way data scientists measure model performance.
Metrics such as AUROC, precision, recall, and calibration are essential during model development because they tell us whether a model performs well. Once an AI solution is deployed, however, executives need to ask a different question:
Is this AI creating measurable value for the organization?
That requires looking beyond technical performance to four broader categories of success.
1. Technical Performance
Every AI solution should continue to be monitored for reliability and performance.
Key measures include:
- Accuracy and calibration
- Data and model drift
- Latency and system reliability
These metrics answer one question:
“Is the model performing as expected?”
2. Workflow Adoption
Even the best model delivers little value if people don’t use it.
Organizations should measure:
- AI utilization and recommendation acceptance
- Time saved and reduction in manual work
- User satisfaction and AI override rates
These metrics answer a more important question:
“Has AI changed the way people work?”
3. Clinical and Operational Outcomes
AI should ultimately improve organizational performance.
Depending on the use case, organizations may track:
- Patient outcomes and care quality
- Throughput and operational efficiency
- Productivity and cost reduction
These metrics demonstrate whether AI is improving care delivery and operational performance.
4. Strategic Business Value
The most mature organizations evaluate AI at the enterprise level.
They ask:
- Is AI generating measurable ROI?
- Is it expanding organizational capacity?
- Is it improving patient and staff experience?
- Is it advancing our strategic priorities?
At this stage, AI is no longer viewed as an isolated technology project—it becomes a strategic organizational capability.
Ultimately, model accuracy is only the beginning of the journey. The true measure of AI success is whether it improves decisions, transforms workflows, and delivers measurable clinical, operational, and financial value.
Looking ahead: AI as an enterprise operating model
Healthcare has reached an important inflection point.
Over the past decade, the conversation centered on whether artificial intelligence worked. Today, the evidence is clear: AI can improve clinical decision-making, optimize operations, streamline administrative work, and enhance access to information.
The question is no longer whether AI works.
The question is how organizations can deploy it responsibly, scale it effectively, and sustain value over time.
This requires a shift in mindset.
Many organizations still approach AI as a collection of independent projects. One team builds a predictive model. Another pilots a generative AI application. A third automates a manual process. While these efforts may deliver local successes, they rarely produce enterprise transformation because they remain disconnected from a broader strategy.
The organizations making the greatest progress are treating AI differently.
They are building an AI operating model.
An effective AI operating model aligns strategy, governance, technology, people, and workflow around a common objective: delivering measurable business value.
That means establishing clear intake and prioritization processes, so AI investments focus on the highest-value opportunities. It means creating governance frameworks that balance innovation with patient safety, privacy, and regulatory compliance. It means investing in data platforms, MLOps, and monitoring capabilities that allow AI solutions to operate reliably in production. Most importantly, it means redesigning workflows so AI becomes part of everyday decision-making rather than another tool employees must remember to use.
Healthcare organizations are also beginning to recognize that AI maturity extends well beyond deploying models. The most advanced organizations continuously evaluate adoption, monitor performance, capture user feedback, measure outcomes, and refine solutions based on real-world experience. AI becomes a continuous improvement capability rather than a one-time implementation.
In my view, this is where healthcare is headed.
The next generation of competitive advantage will not come from building the smartest algorithm or deploying the newest large language model. Those technologies will become increasingly accessible to every organization.
The differentiator will be an organization’s ability to integrate AI into clinical and operational workflows, earn the trust of clinicians and staff, govern AI responsibly, and consistently translate intelligence into action.
The future of healthcare AI is not about replacing physicians, nurses, care managers, analysts, or operational leaders.
It is about enabling every member of the healthcare workforce to operate at the top of their expertise.
Organizations that embrace this philosophy will move beyond isolated AI pilots and build sustainable enterprise capabilities that improve patient care, operational performance, and financial resilience.
Ultimately, the most successful health systems will not be defined by the sophistication of their algorithms.
They will be defined by how effectively they redesign work around human-AI collaboration.
Healthcare’s competitive advantage will not come from having access to better AI models. It will come from building organizations that can integrate AI into everyday workflows, empower their workforce, and continuously translate intelligence into better decisions, better operations, and better patient care.
Executive takeaways
Healthcare organizations are entering a new era of AI adoption. The organizations that achieve lasting value will be those that treat AI as an enterprise transformation initiative rather than simply another technology investment.
As you evaluate your AI strategy, keep these five principles in mind:
1. Start with the workflow, not the algorithm.
Identify operational bottlenecks first. The best AI solutions solve real clinical and operational problems rather than showcasing the latest technology.
2. Measure business value — not just model performance.
Model accuracy is important, but executives should also measure adoption, workflow efficiency, clinician experience, operational impact, financial return, and patient outcomes.
3. Design AI to augment people, not replace them.
The most successful AI systems reduce cognitive burden, surface relevant information, and enable clinicians and operational teams to work at the top of their expertise.
4. Build trust through governance and continuous monitoring.
Responsible AI requires transparent governance, human oversight, performance monitoring, and ongoing evaluation to ensure solutions remain safe, reliable, and aligned with organizational goals.
5. Think beyond pilots — build an enterprise AI operating model.
Long-term success comes from aligning strategy, governance, technology, workflow redesign, change management, and value realization. AI should become a core organizational capability, not a collection of disconnected projects.
Dr. Yapalparvi is an executive leader in artificial intelligence, machine learning and healthcare analytics with more than 15 years of experience leading enterprise AI strategy, and building production-scale AI capabilities, data science and digital transformation across both payer and provider organizations. He has held leadership roles at Mass General Brigham, Optum, Dartmouth-Hitchcock, John Hancock and ANSYS.
At the Becker's 11th Annual IT + Revenue Cycle Conference: The Future of AI & Digital Health, taking place September 14–17 in Chicago, healthcare executives and digital leaders from across the country will come together to explore how AI, interoperability, cybersecurity, and revenue cycle innovation are transforming care delivery, strengthening financial performance, and driving the next era of digital health. Apply for complimentary registration now.
