The state of Utah has been getting a lot of attention for an innovative — yet controversial — pilot project to have AI refill patients’ prescriptions.
Here are 13 things to know about the first-of-its-kind project:
1. In October, the Utah Department of Commerce’s Office of AI Policy approved a year-long pilot where AI from tech company Doctronic autonomously refills prescriptions for 192 common medications for chronic conditions. Utah launched its AI policy office, also a first, in 2024 to allow companies to more freely test AI tools in the state.
2. The service is free for now to patients, who just pay for the medications (Doctronic charges $39 a visit for virtual appointments with human physicians), Marketplace reported.
3. During the first phase of the project, which is still in effect, Doctronic’s human physicians are manually reviewing the AI’s refill decisions before they’re sent to the pharmacy. After 250 refills for each medication group, the state can opt to move to phase two, where physicians vet the AI prescription renewals retroactively for the next 1,000 patients. In phase three, the AI would act largely autonomously, with physicians only doing a spot-check sampling of 5%-10% of cases.
4. In April, the Utah Medical Licensing Board demanded the state stop the experiment, saying it compromises patient safety in the name of “AI or other financial obligations.” The board also said it wasn’t consulted before the technology went live. The commerce department’s Division of Professional Licensing declined to suspend the project but noted that it would modify or cancel the program if safety benchmarks aren’t met.
5. Doctronic, which bills itself as the “first AI system legally authorized to practice medicine” in the U.S. and raised $40 million in March, stated to the media in response: “We are participating in the process as designed, with defined safeguards, physician oversight of every prescription in the first phase of the program, and continued physician involvement throughout. We remain focused on demonstrating safe, evidence-based expansion of routine care access.”
6. Researchers from University of Illinois Urbana-Champaign, Atlanta-based Emory University and Cambridge, Mass.-based Harvard University and Philadelphia-based Penn Medicine also expressed patient-safety concerns in two April editorials in the New England Journal of Medicine and Health Affairs and wondered if the technology qualifies as a medical device that would require FDA approval. A Stanford (Calif.) Medicine professor brought up worries about AI “scope creep” in a March JAMA Health Forum article.
7. In the first few months of the pilot, the AI recommended renewal in 72% of cases — physicians agreed 91% of the time — and escalated the remaining 28% to a human, though physicians found nearly a third of those escalations overly cautious, Doctronic reported in May. “For a new system like this, overcaution is appropriate and welcome,” the AI policy office noted. “In the long term, reducing overcaution without compromising safety would improve patient access to care, but we aren’t rushing to see that happen.”
8. In May, Utah’s AI policy office recommended that the Doctronic AI prescription renewal pilot should strongly encourage — but not require — patients to share their EHR records before receiving a refill, while also ordering the company to fix liability disclaimers in its terms of service and tighten protocols for a handful of higher-risk drugs.
9. Sedative butalbital and arrhythmia medication flecainide were removed from Doctronic’s formulary after the pilot launched, the state reported.
10. The state’s Office of AI Policy OK’d a similar 12-month pilot in April, allowing startup Legion Health’s AI to refill common psychiatric medications.
11. While CIOs told Becker’s they oppose the concept of an independent AI physician — which the Trump administration, a supporter of the Utah experiment, is reportedly discussing creating a regulatory pathway for — some say the idea behind AI refilling prescriptions has merit, either as a way to automate administrative processes for physicians or initiate the licensing process for healthcare AI. The technology “might even do a better job of checking for new side effects or lack of laboratory follow-up testing than a busy human clinician,” noted Curtis Cole, MD, chief global information officer of Ithaca, N.Y.-based Cornell University.
12. The American Medical Association adopted policies June 10 to ensure that AI never replaces physicians’ judgment, calling for “regulations that require AI-enabled technologies to be integrated into physician-led processes.”
13. Doctronic has told media outlets it is in talks to expand its AI prescription refill offering to other states, including Arizona, Missouri, Texas and Wyoming.
At the Becker's 11th Annual IT + Revenue Cycle Conference: The Future of AI & Digital Health, taking place September 14–17 in Chicago, healthcare executives and digital leaders from across the country will come together to explore how AI, interoperability, cybersecurity, and revenue cycle innovation are transforming care delivery, strengthening financial performance, and driving the next era of digital health. Apply for complimentary registration now.