Stanford, OpenAI team up to police healthcare AI agents

Advertisement

Researchers at Stanford (Calif.) Medicine are receiving up to $14.9 million from the Advanced Research Projects Agency for Health to build a monitoring system for autonomous AI agents used in cardiovascular care.

The project, called STEWARD, short for safety, trust, evaluation and watchdog for embedded agentic care delivery, is a three-stage supervisory system. It will monitor AI agents deployed in clinical settings and give on-site reports to care teams. The team also hopes STEWARD will help create a regulatory model for continuous AI monitoring in care delivery.

Roxana Daneshjou, MD, PhD; Euan Ashley, MD; and Akshay Chaudhari, PhD, co-lead the effort, with partners including OpenAI, the University of Washington in Seattle and Tuple Health. The researchers said no systems currently exist for regulatory-grade monitoring of autonomous healthcare AI agents.

“We are excited by the potential of AI agentic workflows to improve patient care in cardiology, but the steps toward safe implementation are not yet clear,” Dr. Ashley said in a Sept. 30 news release.

“Our team will develop AI monitoring tools to ensure that autonomous agents operating in clinical environments remain safe, transparent and accountable,” Dr. Daneshjou said.

The award is part of ARPA-H’s Agentic AI-Enabled Cardiovascular Care Transformation, or ADVOCATE, program, which aims to produce the first FDA-authorized clinical agentic AI system. ARPA-H selected six teams for the four-year, $62.7 million program Sept. 9. Oakland, Calif.-based Kaiser Permanente and Durham, N.C.-based Duke University are leading real-world implementation.

In a separate ARPA-H award, a Stanford Medicine team led by Purvesh Khatri, PhD, received up to $41.4 million to develop an AI-based digital twin of patients’ immune responses to predict outcomes in the ICU.

Advertisement

Next Up in Artificial Intelligence

Advertisement