Health system leaders making the case for AI investment are accustomed to being asked about return on investment. Roberta Schwartz, PhD, chief innovation officer of Houston Methodist, pushes deeper as health system leaders are facing a stark reality: they will have to do their jobs with fewer resources in the future.
In her innovation role, Dr. Schwartz oversees technology strategy across the eight-hospital system and has witnessed a situation familiar to health system executives across the U.S. – a shrinking workforce while patient volume increases.
“All of us have celebrated growing organizations, right? We’re 30,000 people. Now we’re 40,000 people. Celebrating on the back end of that. Now we’re 25,000 people. Now we’re 22,000 people, and we’re servicing more folks with less people with greater access, right? That’s the really, really tough job of leadership right now,” she said. “You can only do it with the innovations. It’s not possible to do it without.”
That’s why Dr. Schwartz holds such a high bar for vendors and partnering companies aiming to grow their relationships with the health system. When vendors approach her about working together, co-developing or bringing their products into the system, she takes a step back and really evaluates each opportunity.
“We require a 3 to 1 or 5 to 1 ROI before we’re going to put in some of that innovation,” Ms. Schwartz said.
The threshold is only part of the standard. The more important question is what counts as a return and that vendors who promise efficiency gains measured in time rather than headcount are not meeting her bar.
“Are you able to tell me I can go down by FTEs? Because if not, and you say to me you’re just giving me back hours. That’s a problem, right?” she said. “We aren’t going to be able to pay for the innovations and keep all the people.”
She has a second concern in how AI gains get measured. She often sees tools that free up staff time but do not reduce costs because the workforce shifts to absorb the return.
“I worry a lot about that because we reallocated [the team] to other areas,” Ms. Schwartz said. “That’s just keeping the cost.”
The standard applies to every vendor who comes to the table. She said she asks directly what headcount reduction the technology has produced.
ROI cannot be evaluated the same way across different types of tools. An OR scheduling project, an ambient scribing platform and an autonomous coding system each require their own measurement approach and their own KPI structure to maintain continued funding.
“It’s different with every single project,” she said. “If you’re looking at them all with one ROI, that’s crazy. You need each one and then the question is for everyone to get funding, how do you have that KPI-type of reporting that says here’s what we tried to achieve and here’s what we’ve done.”
Her skepticism about vendor promises is offset by genuine enthusiasm for AI tools when they perform. Dr. Schwartz described a recent experience building a financial model using Microsoft Copilot that reframed her expectations of the technology.
“I sat down yesterday with me and copilot and within two hours, built a more sophisticated model than I could have ever done in the past,” she said. “It used to have three of us checking to make sure it was right. There was no need to check it.”
The process surprised her.
“It was intimidating to start the prompts and think about it,” she said. “I was like, there’s no way it will understand me and be able to get this right. And indeed, it was flawless.”
