Health systems have spent the past two years asking whether their AI tools work. A more challenging question is now coming into focus: whether the measures they are using to answer that question are still valuable.
Cheristi Cognetta-Rieke, DNP, RN, vice chair of nursing practice transformation at Rochester, Minn.-based Mayo Clinic, spoke to this challenge during a Sept. 14 panel at Becker’s 11th Annual Health IT + Revenue Cycle Conference in Chicago.
Dr. Cognetta-Rieke recalled once being asked at a previous industry event whether the health system’s ambient documentation tool for nurses, co-created with Abridge, had reduced documentation latency.
“I don’t actually care about latency anymore because the conversation is captured,” she said.
For years, nursing teams have tracked documentation latency, or the interval between when a nurse completes a patient assessment and when that assessment lands in the chart. Latency mattered when accuracy depended on nurses’ ability to remember assessment details until they had time to document them, sometimes hours later.
“It doesn’t matter now,” Dr. Cognetta-Rieke said. “The assessment is captured. So latency isn’t a valuable measure of any type of impact or outcome anymore.”
The example points to a broader challenge for executives standing up AI initiatives. Tools that change the underlying work do not simply improve performance against existing benchmarks; they can strip those benchmarks of meaning, leaving leaders without an agreed-upon way to demonstrate value.
New technologies are not just producing different outcomes against the current state, she said. Many are creating an entirely new one.
“What we measure and what matters is also changing,” she said.
This measurement gap is why adoption statistics have become a comfortable place for health systems to stop, according to Dr. Cognetta-Rieke. However, adoption should be the starting point, she said.
“The ending point has to be what is the impact on patients, what is the impact on caregivers, what is the impact on the care team in the ecosystem that’s caring for them,” she said.
At the Becker's 11th Annual IT + Revenue Cycle Conference: The Future of AI & Digital Health, taking place September 14–17 in Chicago, healthcare executives and digital leaders from across the country will come together to explore how AI, interoperability, cybersecurity, and revenue cycle innovation are transforming care delivery, strengthening financial performance, and driving the next era of digital health. Apply for complimentary registration now.
