Health systems across the U.S. are diving deep into AI. They’re launching pilots and scaling the use cases with the best return on investment. They’re also purchasing enterprisewide AI accounts and encouraging their entire team to experiment and innovate with the technology to maximize potential before narrowing down use cases.
“In the next year I expect that AI will shift from pilot to operating assumption and the organizations that are still doing proof of concepts will be lapped by ones treating AI as infrastructure,” said Tomas Gregorio, senior vice president and chief digital information officer at Care New England in Providence, R.I. “You need to get past the pilot stage at this point and that’s why we’re bullish about finding that right company to work with to make sure we can be successful and move the needle slowly as we move forward.”
The opportunities to solve tough problems in healthcare with AI are endless. But so are the expenses.
“In order to successfully use the technology, everybody has to be trained and forewarned of its capabilities,” said Mr. Gregorio. “My team needs to understand the whole billing aspect of how tokens work, and all of that is crazy to figure out. It could lead to exorbitant costs for the organization. We really need to get under the covers and understand what we’re doing. We’re not just releasing a search engine; we’re releasing intelligence to be able to collect information and give you an answer.”
Tokens are the units of data that large language models read and generate. Companies typically provide hundreds of thousands, or millions, of tokens to customers for a fee. But organizations often see tokens disappear quickly depending on how their teams use the technology.
“Education needs to be part of your deployment or else you could see a big bill,” said Mr. Gregorio. “It’s consumption-based and that’ll cripple your organization and not give you the ROI you want.”
It’s a challenge to find the right balance of AI tokens, especially for resource strapped institutions.
“If you go over, you have to pay more. If you don’t use enough, you’ve paid for too many,” said Mr. Gregorio. “If this thing is supposed to be working for you all the time and you aren’t there at night, you’re not using your tokens. You could go crazy because every time you prompt a query, every four characters is a token and it gives you thousands of them, millions of them, but what happens when you surpass it? What are the costs going to be associated when you go over and it slows you down?”
Companies of all sizes and across industries are dealing with the same challenge. Uber recently capped AI use for employees after using its entire annual budget for agentic AI in the first three months of the year, according to The Wall Street Journal. Salesforce is also tracking whether token use drives business outcomes and Google is seeking to reduce AI costs as well after it reported seven-times higher token use year over year.
AI use is increasing at many health systems and the payment models charged by vendors is also evolving. Companies that charge a more reasonable price for their services are starting to see the value they’re bringing and beginning to charge more. Houston Methodist has identified users who may need additional capabilities and designed accounts accordingly.
“We’ve been allowing certain individuals in our organization to utilize Microsoft Copilot to see how they use it to create their own agents, but it’s in pilot mode,” said Michelle Stansbury, associate chief innovation officer and vice president of IT applications at Houston Methodist. “We are trying to figure it out. You don’t want to open it up to everyone because then you really have to figure out what’s the cost, and we have made the decision that we will not be building agents unless it truly has a return for us. If we’ve built one and within a certain timeframe we see that it’s doing nothing but just adding cost to the organization, we’ll shut it off. It’s making people be very thoughtful in the use cases they are developing.”
Team members developing agents are asked to document their processes and collect data points to determine whether it makes sense to build or buy the technology.
“There is so much opportunity out there with overall agentic AI, but it’s having to make sure that it makes sense without opening the floodgates and letting everybody go crazy with it,” she said.
The projects most attractive to health systems are often those improving patient outcomes or the patient experience. AI-driven applications that support clinician and staff efficiency are also in high demand even as the costs associated with AI mount. But they may not be astronomical forever.
“I do think it is kind of a wild and crazy moment,” said Ms. Stansbury. “I think it’s going to land somewhere in the middle between companies undercharging and overcharging. I think it will come back to something reasonable.”
At the Becker's 11th Annual IT + Revenue Cycle Conference: The Future of AI & Digital Health, taking place September 14–17 in Chicago, healthcare executives and digital leaders from across the country will come together to explore how AI, interoperability, cybersecurity, and revenue cycle innovation are transforming care delivery, strengthening financial performance, and driving the next era of digital health. Apply for complimentary registration now.