As Chicago-based Lurie Children’s prepares artificial intelligence tools for wider clinical use, the organization “locks” the models, prompts and outputs, according to Rajiv Kolagani, chief data and AI officer at Lurie Children’s.
The practice is part of an internal framework Mr. Kolagani calls “AI Trust by Design.” He told Becker’s the framework is intended to build guardrails around generative AI, which can make mistakes, and increase confidence in its outputs before broader clinical use.
As part of the process, Lurie Children’s checks whether a model is using the intended source data, compares the outputs of two models run against the same data and uses a third large language model to assess whether the first two are working. The organization also checks whether the same data produce the same output over time to monitor for drift.
“For us, when two AIs agree, that is a very good sign that the output is correct,” he said.
Mr. Kolagani described that agreement as a confidence factor rather than the only form of review. Clinicians also evaluate the technology using complex cases and patient records, identify missing information and help refine the prompts. He said manual verification becomes especially difficult when a patient record contains 20,000 notes. That manual burden is part of why Lurie Children’s is applying its validation framework to two tools now moving through smaller-scale deployment.
One is a hospitalwide Condition Lens product, also called Intelligent Chart Assist. The tool references infection definitions in a National Healthcare Safety Network safety manual, reviews a patient’s chart and guides users through a questionnaire to assess whether an event is an infection.
The tool is intended to reduce the reporting and data-exploration burdens placed on clinical teams, according to Mr. Kolagani.
Another tool, Flight Plan, is being developed generally for cardiology clinicians. Mr. Kolagani said some patients stay 300 to 600 days, leaving clinicians with extensive records to review.
The tool uses AI to pull out critical information, build clinical trajectories, identify opportunities to improve documentation and help clinicians examine factors related to clinical outcomes. Mr. Kolagani said clinicians can interact with the AI utility instead of spending 10 to 12 hours reviewing a chart.
Neither tool has been deployed at scale. Mr. Kolagani said Lurie Children’s had been testing its AI work extensively over the previous three months and was targeting the end of August to finish deployment testing and be able to deploy at scale.
The testing approach sits within a broader governance process involving operational and technology leaders, as well as risk, legal and compliance teams. Mr. Kolagani said clinicians and other users are also involved in evaluating a tool’s usability, utility and effects on their workflows.
“We don’t build AI in isolation. We build it as a team,” he said.
Lurie Children’s is also investing in AI education as it prepares for wider adoption.
The organization scheduled an enterprise-wide AI summit for Aug. 27 covering AI foundations, essential concepts, policy, ethics, available tools and the technology’s future. More than 200 people have registered to attend in person, Mr. Kolagani said.
The summit is part of a broader AI literacy effort that also includes a weekly newsletter featuring AI tips, lunch-and-learn sessions and hands-on training.
“There’s a whole bunch of AI evangelism that we’re doing that’s really working for us,” Mr. Kolagani said.
He said those investments in governance and education are intended to help employees with differing levels of familiarity and comfort understand both AI’s potential benefits and its risks.
“I think most people look at governance as a policing function, but we look at it as a catalyst,” Mr. Kolagani said.
At the Becker's 11th Annual IT + Revenue Cycle Conference: The Future of AI & Digital Health, taking place September 14–17 in Chicago, healthcare executives and digital leaders from across the country will come together to explore how AI, interoperability, cybersecurity, and revenue cycle innovation are transforming care delivery, strengthening financial performance, and driving the next era of digital health. Apply for complimentary registration now.
