Hospitals are increasingly paying for artificial intelligence by the token, turning a once-obscure technical term into a line item finance teams are starting to watch.
Tokens are the basic unit large language models use to read and generate text. A model breaks words and phrases into smaller pieces, sometimes whole words, sometimes fragments, converts them into numbers and processes them one at a time. Most AI vendors bill by the token, counting both the text fed into a model (the prompt) and the text it produces (the output). Every four characters is roughly a token, Tomas Gregorio, senior vice president and chief digital information officer at Care New England, said in a June interview with Becker’s, meaning a single query can generate thousands, or millions, of tokens before a health system notices.
For health systems, that billing unit is becoming a budget line. Ambient AI scribes, EHR copilots and revenue cycle tools all run on token consumption behind the scenes, even when a vendor sells a flat per-provider subscription.
“It’s consumption-based and that’ll cripple your organization and not give you the ROI you want,” Mr. Gregorio said.
Epic’s Art, Emmie and Penny tools, for example, run on Microsoft’s Azure OpenAI service, and Epic has said more than 85% of its customers are now using its AI tools.
Some health systems are already restructuring access to manage the exposure. Houston Methodist has limited Microsoft Copilot to select employees testing how to build their own agents, and the system will shut off any agent that adds cost without a clear return, said Michelle Stansbury, associate chief innovation officer and vice president of IT applications there.
The problem isn’t unique to healthcare: Uber capped AI use for employees after burning through its entire annual agentic AI budget in three months, Salesforce is tracking whether token use drives business outcomes, and Google has reported seven-times higher token use year over year.
The token model matters for CIOs and CFOs because cost scales with use. A longer patient note, a wider context window or a more complex model all mean more tokens, and more tokens mean a bigger bill.
As health systems move AI from pilots to system-wide rollouts, usage-based token pricing is becoming a line item finance teams are watching alongside vendor contracts, EHR integration and compliance costs. As more clinical and administrative work moves onto AI, understanding the cost of AI tokens is becoming as important to health IT leaders as understanding subscription pricing used to be.
At the Becker's 11th Annual IT + Revenue Cycle Conference: The Future of AI & Digital Health, taking place September 14–17 in Chicago, healthcare executives and digital leaders from across the country will come together to explore how AI, interoperability, cybersecurity, and revenue cycle innovation are transforming care delivery, strengthening financial performance, and driving the next era of digital health. Apply for complimentary registration now.
