The AI industry’s leaders spent the past month warning that their models are gaining capabilities faster than anyone can verify they’re safe. Health systems are deploying versions of that same technology in places where mistakes reach patients.
The warnings began with a September essay from Anthropic CEO Dario Amodei, PhD, urging the tech industry to “pace the frontier,” deliberately slowing how quickly AI models gain new capabilities. OpenAI CEO Sam Altman and Elon Musk quickly voiced support.
Health systems don’t build frontier models, but they are putting AI to work in clinical documentation, patient communication and back-office operations. Have they modified their adoption or deployment plans?
“Intermountain Health has not changed the pace of our AI strategy in response to recent calls to slow down the advancement of frontier models,” said Ryan Smith, senior vice president and chief digital and information officer of the Salt Lake City-based 34-hospital system. “In many ways, our system, and the healthcare industry as a whole, have been operating with AI guardrails long before the current debate emerged.”
John Brownstein, PhD, chief innovation officer of Boston Children’s Hospital, said there hasn’t been a slowdown at his organization, which has been balancing the swiftness of adoption against worries about AI’s risks and its environmental impact.
“We’re trying to meet people where they are,” he said. “We understand that there are real concerns about the speed, while at the same time, there’s a huge amount of demand.”
Brett Oliver, MD, chief medical information and AI officer of Louisville, Ky.-based Baptist Health, said healthcare has traditionally moved more deliberately than consumer tech anyway.
“We are not operating at the pace of Dario Amodei, Sam Altman or Mark Zuckerberg when it comes to deploying new technologies,” he said. “At Baptist Health, our approach has always been measured and governance-driven. … As a result, the calls for greater caution from AI industry leaders largely reinforce principles that were already part of our governance framework.”
Deepti Pandita, MD, chief medical informatics and AI officer of Orange, Calif.-based UCI Health, gave a similar answer. “In healthcare, we’ve always needed to balance innovation with patient safety, privacy, and rigorous governance,” she said.
There’s also a more basic mismatch between the warnings and healthcare’s reality. Dr. Amodei’s proposal targets how quickly the most capable models gain new abilities — models that, in many cases, are still being trained inside the labs. Health systems work further down the line.
“We do two things: 1) focus keenly on evaluation and 2) hope to catch up with the latest models,” said Nigam Shah, MD, PhD, chief data scientist at Stanford Health Care. “I say that because while the frontier labs debate pausing development of unreleased models, health systems are several development cycles behind in terms of what models they use.”
Stanford Health Care is still using GPT-5.4 generation models, Dr. Shah noted. OpenAI recently shelved GPT-6.1 Astra over concerns about deceptive behavior.
Most large health systems have spent the past several years building formal AI oversight: committees that review new tools, vet vendors and monitor models after they go live.
“At Stanford Health Care, because we have always used a framework to evaluate AI tools (FURM) as part of our responsible AI lifecycle, we have not had to speed up or slow down adoption or deployment,” said Michael Pfeffer, MD, senior vice president and chief information and digital officer of the health system. “We identify problems that we would like to solve that enhance our mission areas of patient care, research, and education, and if the solutions to those problems require AI, then we follow our governance process.”
FURM, which Stanford developed to assess whether AI models are fair, useful and reliable before they are deployed, is one version of a gatekeeping process that exists elsewhere in healthcare.
“Every novel AI tool and new use case for existing tools, across clinical, research, and operational settings, undergoes a rigorous governance process,” said Rebecca Mishuris, MD, vice president and chief health information officer of Somerville, Mass.-based Mass General Brigham. “We take a phased approach to any AI application that can have an impact on patient care, assessing safety, effectiveness, and potential benefits before broader implementation.”
At Philadelphia-based Penn Medicine, the debate sent leaders back to that same foundation.
“The recent discussion about slowing the pace of AI has reinforced the importance of what we’re already doing at Penn Medicine around governance, oversight, and responsible implementation, which provides us with confidence in our continued approach to this work,” said Mitchell Schnall, MD, PhD, senior vice president for data and technology solutions at the academic system.
The warnings have also reached healthcare’s top leadership. Days after Dr. Amodei’s essay, a guest columnist wrote in Becker’s urging hospital and health system boards to scrutinize both the doomsday predictions and management’s own AI strategies. At Boston Children’s, the headlines have been part of those conversations.
“These elicit senior leadership conversations, board conversations, absolutely, and they force us to think very critically about how we’re meeting this moment in time,” Dr. Brownstein said.
He stopped short of calling the warnings the cause of added investment, but said they have helped make the case for it.
“There is an urgency to investing in our systems and the people and the security, and so because these issues are really taking center stage, it’s actually in some ways at least a forcing function for more resourcing, and an enterprise operating model when it comes to AI,” he said.
Mr. Smith offered a different read on Intermountain’s boardroom.
“Our board, executive leaders, and clinical leaders are deeply engaged in AI discussions, but the conversation is typically bigger than any one industry headline,” he said. “The questions our leaders ask are remarkably practical.”
If there is one place the labs’ safety worries and health systems’ plans meet, it is agentic AI — systems that can take actions on their own rather than only generating content or predictions.
“Agentic AI presents exciting possibilities, but it also raises important questions around authority, transparency, escalation and oversight. Those are areas we are actively evaluating as the technology matures,” said Chif Umejei, senior vice president and CIO of New York City-based NewYork-Presbyterian.
“Our approach is consistent with how we evaluate every technology: Autonomy has to be earned. Systems should prove they are safe, measurable and trustworthy before they take on greater responsibility,” Mr. Smith said.
“The goal of governance is not to create barriers. The goal is to create confidence necessary to scale.”
Becker’s 3rd Annual Fall Chief Pharmacy Officer Summit is the top tier event that the people leading the future of pharmacy convene. This 3-day event is built specifically for Chief Pharmacy Officers and senior pharmacy leaders. The sessions examine the workforce, financial and clinical priorities shaping pharmacy today. Becker’s is where candid discussions flourish and new ideas thrive. Becker's 3rd Annual Fall Chief Pharmacy Officer Summit
