About three years ago, Norfolk, Va.-based Sentara established its AI steering committee and governance group to guide ethics and combat bias in AI as they began incorporating artificial intelligence into the acute care setting.
Sentara started a pilot examining how AI could make the EHR more powerful. The health system provided hospitalists with an engine that would look through the entire EHR and history of all the patient’s data and identify possible conditions for the clinicians to look out for. The engine provided options and the hospitalist could agree or disagree with the engine’s assessments. The information they agreed with was included in the patient’s notes and gave the hospitalists more time to interact with the patient.
The pilot has had broader implications as well.
“The bigger benefit was, after we started training the large language model engine we were going to use for discharge summary at the end of the episode of care it would take all the historical data that was gained during the hospital stay and put it in a very specific, targeted discharge summary to leverage the post-acute care with providers that were going to continue that person’s care once they were discharged,” said Tim Skeen, CIO of Sentara Healthcare. “It was an interesting journey, not just to get adoption and be able to figure out how to work with this assistance and trust it, and then to challenge it to make them more effective in their job.”
Early on the technology did have some hallucinations with the discharge notes. Mr. Skeen’s team continued training the model to avoid those issues and achieve the level of quality they wanted with their notes every time.
“We found that to be very beneficial in terms of proving quality, safety and financial benefits in terms of being able to fully build the work that was done more comprehensively, and it’s something we’ve rolled across to all of our 12 acute facilities,” said Mr. Skeen. “Now we’re looking to expand it to other modalities and other sites of service.”
Moving from the pilot to full rollout was a team effort and required engagement from leaders across the health system.
“This isn’t about the technology and it’s not about telling clinicians how cool the technology is,” said Mr. Skeen. “It’s about getting the right champions within the use case to say ‘how is this going to make your job better?’ Luckily, my chief medical informatics officer Dr. Joe Evans was the one championing this and being that voice as opposed to the CIO voice, which the clinicians in the hospital wouldn’t listen to. But the CMIO, they’ll listen to him because he’s a physician as well.”
Dr. Evans then established champions at each facility to drive adoption and results. Mr. Skeen said it was important to pick the right facility to roll out the technology to first, with a president who would support the physician champions reimagining workflow and processes changes.
“The intuitiveness of the technology was a benefit, but getting that champion across the first two facilities, and then doing that one at a time for the others to make sure we had the champions was really important,” said Mr. Skeen. “I’m happy to say we’re well over 75%-plus adoption with this technology. With a lot of historical experience I’ve had, getting even 25% adoption from clinicians would make me happy.”
The champions sold the technology to their physician teams and built momentum for the rollout, and then actually seeing how the technology reduced burnout and gave them more time with patients was an additional incentive. Mr. Skeen and his team have taken the physician champion approach to other AI and ambient AI rollouts since then.
Looking ahead, Sentara has a strong process for selecting, piloting and rolling out future AI solutions related to their multi-year transformation, One Sentara. The One Sentara framework views the organization across all care delivery sites and the health plan; the organization will consider how technology affects the patients, members and healthcare consumers. The technology with the biggest potential positive impact will win out.
“Historically, the person who screamed the loudest would get the capital funding as opposed to having a good business case and ROI, much less measuring whether the ROI they sold on the business case actually came to fruition,” said Mr. Skeen. “Within that framework, within One Sentara, we create a whole process and governance tool set that basically has the entire portfolio captured in it.”
The model records the process from ideation through implementation and the final realization of value to decide whether to move forward with funding. The process requires building a strong business case and working with the finance department as a partner to sign off on the plan. Then the finance team measures the ROI for five years to make sure the results continue to generate value.
Sentara’s model is particularly beneficial as new generative AI and predictive models become available because the ROI is soft, focused on cost avoidance and reducing clinician burnout. But there is a hard ROI to measure. By improving the note process for the whole episode of care, the health system is collecting more revenue because the tool supports documentation and ensures the team is billing for the entire episode.
The governance model has pushed innovators to identify the hard ROI and ensure successes. It’s also helped the team embrace the “fail fast” mentality because Sentara is moving quickly to achieve results.
“You have a belief that the technology is going to solve this ROI, but if it doesn’t, you pivot and shut it down,” said Mr. Skeen. “Let’s get those resources and grab the next thing that has the biggest ROI that we want to bring in that brings value to our business as well as the consumer.”
