Researchers at the Icahn School of Medicine at Mount Sinai in New York City are examining how artificial intelligence systems perform when faced with heavy workloads in clinical environments, finding that system design may play a larger role than the underlying technology itself.
A study published March 9 in npj Health Systems evaluated whether distributing clinical tasks among multiple specialized AI “agents” could improve performance compared with relying on a single, all-purpose system. The findings suggest a coordinated, multi-agent approach can maintain accuracy while reducing computing demands as workloads increase, according to a March 10 news release from the Icahn School of Medicine.
Here are six key findings from the study:
- The researchers compared two AI architectures: a single system responsible for handling multiple clinical functions and a network of specialized agents overseen by a central coordinating system, often referred to as an “orchestrator.” Using large language models, the team tested the systems on common clinical tasks such as retrieving patient information, extracting data and calculating medication doses.
- The simulations replicated real-world conditions in which multiple tasks arrive simultaneously. The systems were tested under workloads of up to 80 concurrent tasks.
- The multi-agent model maintained higher accuracy while using significantly fewer computing resources than the single-agent system, according to the release. In some cases, the coordinated system required up to 65 times fewer computing resources.
- The study simulated clinical “traffic,” where multiple types of tasks arrive at once and compete for attention. Under those conditions, the single-agent system’s accuracy fell to 16% when managing 80 simultaneous tasks while using far more computing resources, according to the release.
- Researchers said the orchestrated approach may also improve transparency in how AI decisions are produced. Each step of the process can be logged and traced, which could help clinicians and health systems identify errors or better understand how an answer was generated.
- The research team plans to test the multi-agent system directly in clinical settings using real-time patient data. If successful, the approach could help hospitals and health systems scale AI tools to handle peak workloads without sacrificing safety or performance.
At the Becker's 11th Annual IT + Revenue Cycle Conference: The Future of AI & Digital Health, taking place September 14–17 in Chicago, healthcare executives and digital leaders from across the country will come together to explore how AI, interoperability, cybersecurity, and revenue cycle innovation are transforming care delivery, strengthening financial performance, and driving the next era of digital health. Apply for complimentary registration now.