Hasan Ahmad, MD, vice president and associate chief medical information officer at Fort Wayne, Ind.-based Parkview Health, spent years untangling a problem that had nothing to do with AI: interruptive EHR alerts that had piled up for 15 years with no one able to say who owned them or why they existed.
“The problem was never the alert,” Dr. Ahmad recalled last week at Becker’s IT + Revenue Cycle Conference in Chicago. “It was that these things got built over time and nobody went back to look at them. … It’s very hard to turn something off because you need to figure out who owns it, why it was there. These things were never documented anywhere.”
That experience is now shaping how Parkview approaches AI before a tool goes live.
“We really need to have that clear answer of the kill switch. It’s not optional, it’s part of the design requirements,” Dr. Ahmad said. “We have to determine upfront what is … the metric that we’re trying to move … and agreement up front about what does failure look like in order to turn things off, so that we don’t have to have extended negotiation about it.”
He compares the discipline to how a pharmacy handles a new drug. “When they have new medication, they have an intake process, they have a monitoring process, and then they have a process for getting rid of stuff,” he said. “Apply those same principles to your AI inventory.”
Micheal Sweet, CIO of Brigham and Women’s Hospital in Boston, said the switch gets built before launch, not after a problem shows up.
“What is our kill switch? When do we decide we’re gonna take that out?” Mr. Sweet said. “We come up with a date in the beginning. We say once we hit these metrics, this is the date support stops. You need to decide whether it’s going to go into production or it’s going to be shut down … so they don’t run forever.”
Nari Gopala, chief digital officer at Rochester, Minn.-based Mayo Clinic, said the switch is only as good as knowing who’s allowed to pull it. He breaks AI governance into three separate questions before a tool ever reaches a patient.
“Who’s building the technology? Who’s actually making the decision that this technology is ready for a particular provider or patient-facing use case … and then who actually owns the accountability over the operation of that AI — who’s getting that 2 a.m. call if the model misbehaves,” Mr. Gopala said.
For Maggie Helms, senior vice president and chief data, AI and digital officer of Bloomington, Minn.-based HealthPartners, the kill switch isn’t only a per-tool decision — it’s a line drawn around an entire category of use.
“We’re still not willing to compromise on it,” Ms. Helms said, referring to letting AI agents adjudicate clinical disagreements on their own. “If we create this framework where you do have agents reckoning with each other and making clinical decisions … I think we’re not doing our jobs as healthcare practitioners. So we are occasionally unpopular, but still trying to stay slow, steady and keep quite grounded in trust.”
Mark Lo, MD, associate chief medical information officer at Seattle Children’s, said how much kill-switch rigor a tool gets depends on a score, not a gut check. His team rates every AI use case on reach, whether a human stays in the loop, how reversible its actions are, and potential harm.
“We score it out and then see how it’s going to fall out, and then that will determine whether this is something we could potentially just release,” Dr. Lo said, “or whether this is a full-scale pilot where we’re going to take metrics and actually have a very clear go, no-go threshold.”
Chris Gay, CEO of business-to-business insurer Evry Health, built his own AI code-review layer after generative tools flooded his engineers with bloated, unreviewable code.
“You have to embrace the robot army,” Mr. Gay said. “On the flip side, you can’t trust your robot army.” His team wrapped its systems in what he called an observability layer instead of taking tools on faith. “In the past six months, this approach yielded 20% of our pull requests — in our organization, we found unknown security vulnerabilities,” he said.
