The list of what artificial intelligence promises to fix in healthcare is long: administrative burden, diagnostic accuracy, care coordination, staffing shortages. But the things it could break, according to executives at major health systems, deserves equal attention.
When asked to name the most dangerous trend in healthcare today, leaders from systems including Baystate Health, Banner Health and Cincinnati Children’s Hospital pointed to AI — not as a technology to reject, but as a force whose adoption is outpacing the governance, human investment and cultural change required to make it work safely. Their concerns gather around three themes: the erosion of human judgment and connection, inadequate accountability frameworks and the workforce consequences of deploying AI without deliberate reskilling.
“Artificial intelligence is the most dangerous trend in healthcare today,” Peter Banko, president and CEO of Springfield, Mass.-based Baystate Health. “Better said, the promise of AI is the most dangerous trend. Vendors are peddling potential partnerships using AI to transform literally every part of our clinical enterprise and business. AI can be a most profound catalyst (in the order of magnitude of the Industrial Revolution) that can help us solve massive systemic challenges. Before taking that dangerous lunge, we need to understand and put frameworks into place around the governance, ethical, and emotional impact of AI on our industry.”
That concern — adoption running ahead of accountability — is shaping how Steve Davis, CEO of Cincinnati Children’s Hospital, thinks about the workforce implications of AI-enabled care.
“The most dangerous trend in healthcare today may be the assumption that AI adoption alone will improve performance, without equal investment in the human judgment, analytical skill, and training required to use it safely and effectively,” Mr. Davis said. “AI can strengthen decision-making, but only if organizations pair adoption with deliberate development of the critical thinking and analytical skills needed to use these tools safely and effectively.”
Mr. Davis warned that redeploying staff into AI-enabled workflows without adequate support risks weakening the very capabilities those tools are meant to augment. Organizations will need thoughtful reskilling and upskilling plans for employees whose roles are being reshaped by AI.
“The goal should be to use AI as a tool that amplifies human expertise, not intentionally diminishes it,” said Mr. Davis.
The risk of diminishment extends beyond the workforce. Joe Avelino, CEO of College Medical Center in Long Beach, Calif., said the danger he watches most closely is the substitution of technology for the human relationships at the heart of care.
“AI will undoubtedly become an increasingly important tool to help us identify patients at risk, improve access to care, streamline administrative burdens, and support better clinical decision-making,” Mr. Avelino said. “However, AI cannot replace the healing power of a compassionate physician, a reassuring nurse, a dedicated social worker, or a sitter who simply sits quietly with someone during their darkest moment.”
Mr. Avelino’s point speaks to what patients carry away from a care encounter long after the clinical details fade.
“Our patients may not always remember their diagnosis or the medication we prescribed but they will remember how we made them feel and whether they were treated with dignity, respect, and compassion,” he said.
Benjamin Schwartz, MD, president of academic delivery at Phoenix-based Banner Health, framed the risk in structural terms. Technology and research capabilities are advancing quickly while the systems built to pay for and deliver care are growing more complex and both can conspire to distance clinicians from the individual patient.
“The most dangerous trend in healthcare today is that we lose the human element,” Dr. Schwartz said.
At Banner, a partnership with the University of Arizona is designed to keep the system grounded in that reality, Dr. Schwartz said — training the next generation of providers to understand that technology and complex systems should facilitate community health, not complicate it.
“It can all start to feel very abstract and removed from the actual human sitting in the exam room, the very person all of this is built to serve,” he said. “Technology and our research capabilities are advancing fast, with things now possible that have never existed before. The business of healthcare, how we pay for and deliver care, is growing more complex by the day. It can all start to feel very abstract and removed from the actual human sitting in the exam room, the very person all of this is built to serve.”
Banner partners with the University of Arizona to stay “grounded in reality,” said Mr. Schwartz.
“We are not just pushing the boundaries of medicine together, we are training the next generation of providers to understand that technology and complex systems should facilitate community health, not complicate it,” he said. “The goal has always been simple: make sure we provide the best care to the patients and communities we serve.”
Matthew Fry, president and CEO of Freeman Health System in Joplin, Mo., named AI as simultaneously the field’s greatest opportunity and its greatest risk. He pointed to a consequence that reaches well beyond the hospital walls.
“We’re already seeing how reliance on large language models can shape whether people seek medical attention at all, or whether they trust the advice of the professionals trained to give it,” Mr. Fry said. “As these tools become part of everyday life, it falls to all of us — healthcare providers included — to keep our audiences grounded in timely information so they can make sound decisions about their health.”
Freeman has invested in upgrading its EHR system and patient portal to ensure patients can access care information and connect with providers from home. But the drive to innovate has a hard limit.
“In the race to innovate, we can’t forget that healthcare is personal,” he said. “The empathy and judgment a provider brings to a patient relationship can’t be replicated by a model, and it shouldn’t be.”
The efficiency pressures that make AI adoption so appealing are, in some cases, already generating costs before AI enters the picture. Angelo Milazzo, MD, chief medical officer of Duke Health Integrated Practice in Durham, N.C., and chief clinical officer of North Carolina Children’s Hospital and Health System, said the erosion of face-to-face time with patients, driven by throughput demands, is troubling.
“Time is a precious commodity, and it takes time with patients — focused, intentional, purposeful time — to effectively evaluate and manage disease,” Dr. Milazzo said. “Most diagnoses are made through obtaining a thorough history, which requires a meaningful conversation without the constraints and pressures of a too-short office visit.”
At the Becker's 11th Annual IT + Revenue Cycle Conference: The Future of AI & Digital Health, taking place September 14–17 in Chicago, healthcare executives and digital leaders from across the country will come together to explore how AI, interoperability, cybersecurity, and revenue cycle innovation are transforming care delivery, strengthening financial performance, and driving the next era of digital health. Apply for complimentary registration now.
