Utah’s experiment with letting AI handle part of a physician’s job grew considerably this month. The state approved what one company calls the nation’s first AI to write new prescriptions, gave two Salt Lake City-based health systems a fast track for future AI pilots and enlisted outside evaluators to check companies’ claims.
Those moves build on a first-of-its-kind — and controversial — pilot that has had AI renewing prescriptions in the state since late 2025.
What exactly is Utah doing, though?
The Utah Department of Commerce runs an Office of Artificial Intelligence Policy, which the state launched in 2024 as the first office of its kind. The office can sign temporary agreements that give companies regulatory relief to test AI tools under set guardrails. In exchange, companies send the state monthly reports on how the tools perform, and the office can require changes or end a pilot if risks emerge.
Where it started
The first prescribing pilot came from Doctronic. The office approved a yearlong pilot in October 2025 letting the company’s AI renew prescriptions for 192 common medications for chronic conditions, and the state publicly announced it Jan. 6. The AI cannot issue new prescriptions, change treatment plans or renew controlled substances.
Oversight steps down in three phases. In the first, Doctronic’s physicians review every AI refill decision before it goes to the pharmacy. The state revised the pilot so each medication group can move to phase two on its own once Doctronic has filled 250 prescriptions in that group, rather than after 250 refills total; any move still requires the office’s approval. In phase two, physicians review the AI’s renewals retroactively for the next 1,000 patients. In phase three, the AI would act largely autonomously, with physicians spot-checking 5% to 10% of cases.
In April, the office OK’d a similar 12-month pilot allowing startup Legion Health’s AI to refill common psychiatric medications.
Early results
In the first few months of the Doctronic pilot, the AI recommended renewal in 72% of cases — physicians agreed 91% of the time — and escalated the remaining 28% to a human, though physicians found nearly a third of those escalations overly cautious, Doctronic reported in May.
“For a new system like this, overcaution is appropriate and welcome,” the AI policy office said. “In the long term, reducing overcaution without compromising safety would improve patient access to care, but we aren’t rushing to see that happen.”
The state has also tightened the program as it goes. The sedative butalbital and arrhythmia medication flecainide were removed from Doctronic’s formulary after launch. In May, the office recommended that Doctronic strongly encourage — but not require — patients to share their EHR records before receiving a refill, while also ordering the company to fix liability disclaimers in its terms of service and tighten protocols for a handful of higher-risk drugs.
As of the state’s most recent public update, the Doctronic pilot remained in phase one and no serious safety incidents had been reported. A third-party red-teaming report identified potential vulnerabilities under adversarial use, though the office said the pilot’s guardrails “substantially limit the likelihood of harm to typical users.”
The pushback
Not everyone in Utah’s medical establishment has been on board. In April, the Utah Medical Licensing Board demanded the state stop the Doctronic experiment, saying it compromises patient safety in the name of “AI or other financial obligations” and that the board wasn’t consulted before the technology went live. The Commerce Department’s Division of Professional Licensing declined to suspend the project but said it would modify or cancel the program if safety benchmarks aren’t met.
Doctronic, which raised $40 million in March, responded in a statement to the media.
“We are participating in the process as designed, with defined safeguards, physician oversight of every prescription in the first phase of the program, and continued physician involvement throughout. We remain focused on demonstrating safe, evidence-based expansion of routine care access,” the company said.
Researchers from University of Illinois Urbana-Champaign, Atlanta-based Emory University, Cambridge, Mass.-based Harvard University and Philadelphia-based Penn Medicine also expressed patient-safety concerns in two April editorials in the New England Journal of Medicine and Health Affairs and questioned whether the technology qualifies as a medical device that would require FDA approval. A professor at Palo Alto, Calif.-based Stanford Medicine raised concerns about AI “scope creep” in a March JAMA Health Forum article.
The American Medical Association adopted policies in June to ensure that AI never replaces physicians’ judgment, calling for “regulations that require AI-enabled technologies to be integrated into physician-led processes.”
From refills to new prescriptions
Utah widened the experiment Oct. 5, when the Commerce Department added a healthcare pillar to its Pro-Human AI Initiative and approved five new agreements.
One of them lets AI from Nolla Health issue initial acne prescriptions without a physician visit. The company says it is the first in the U.S. cleared for AI to write new prescriptions rather than renew existing ones. Utah residents 18 and older pay $4.99 a month, complete a 10- to 15-minute questionnaire and a face scan, and get a prescription within minutes. The AI can only choose from a short list of physician-approved topical treatments and must send patients it can’t confidently treat to a physician.
A physician approves every Nolla prescription for the first 100 patients. Through the first 500, the AI prescribes on its own and a physician reviews its decisions daily after the fact; after that, a physician reviews a weekly sample. Clinicians agreed with the AI’s recommendations in more than 96% of real-world cases, according to Nolla, which also worked with the Utah Medical Licensing Board before launch.
August AI won approval for a chronic-medication renewal pilot that charges $4 per refill. At launch, a Utah-licensed clinician reviews every decision, and only the lowest-risk medications — such as frontline treatments for diabetes, high blood pressure and high cholesterol — are eligible. Controlled substances, injectables, antibiotics, new medications and dose changes are excluded. Like Doctronic’s, the pilot is designed to end with physicians spot-checking a small share of decisions — 5% in August’s case — but each reduction in oversight requires validation from independent evaluators and the state’s approval, the company said.
Health systems get a fast track
Salt Lake City-based Intermountain Health and University of Utah Health received umbrella agreements that create expedited review pathways for future healthcare AI pilots, though each pilot still must be vetted individually. The state also approved Expect Fitness to test AI-enabled physical therapy for women dealing with postpartum recovery, urinary incontinence and pelvic health issues.
The new agreements come with common ground rules. Participants must disclose AI use to patients, carry specialty medical malpractice insurance, maintain 24-hour incident response plans and preserve patients’ traditional legal remedies. Six independent evaluators will assess companies’ technical claims, how accurately they report AI decisions and whether they align with medical standards, and pilot data will be shared with lawmakers and the public.
What health system leaders think
While CIOs told Becker’s they oppose the concept of an independent AI physician — which the Trump administration, a supporter of the Utah experiment, is reportedly discussing creating a regulatory pathway for — some say the idea behind AI refilling prescriptions has merit, either as a way to automate administrative processes for physicians or to start the licensing process for healthcare AI.
Curtis Cole, MD, chief global information officer of Ithaca, N.Y.-based Cornell University, said the technology “might even do a better job of checking for new side effects or lack of laboratory follow-up testing than a busy human clinician.”
What’s next
Doctronic’s agreement runs through October 2026, with the option for the state to review and renew it for another year. August has not set a public launch date for its pilot. And the experiment may not stay in Utah: Doctronic has told media outlets it is in talks to expand its AI prescription refill offering to other states, including Arizona, Missouri, Texas and Wyoming.
