As health systems expand their use of artificial intelligence, some are asking a more fundamental question about vendor access to healthcare data: Does the data need to leave the health system at all?
That is the approach Palo Alto, Calif.-based Stanford Health Care is taking as it evaluates AI technologies, Christian Lindmark, vice president and chief technology officer, told Becker’s.
“For us, the more fundamental question isn’t how much data a vendor needs, but whether the data needs to leave our environment at all,” Mr. Lindmark said.
Stanford prefers to bring a vendor’s model into its own environment rather than send health system data to a third party, allowing the organization to test and validate the technology using its own data while keeping that information within Stanford’s environment.
When that approach is not possible, Stanford limits vendor access to the specific task, favors deidentified or limited datasets and requires vendors to justify requests for broader access.
“Our preferred move is architectural: contain the model, not just the data request,” Mr. Lindmark said.
Other health systems are similarly applying “minimum necessary” principles as AI tools gain access to more clinical and enterprise information.
At Children’s Hospital of Philadelphia (CHOP), AI tools used directly by employees operate under role-based access controls, meaning the technology can only access information the user is already authorized to see, Hojjat Salmasian, vice president and chief data and analytics officer, told Becker’s.
For automated AI workflows, CHOP limits data sharing to information required for the intended purpose. The policy is part of the organization’s broader enterprise guidance for data sharing, which also applies to research and regulatory reporting.
Wilmington, Del.-based ChristianaCare takes a similar approach. Anahi Santiago, chief information security officer, told Becker’s that vendors are given access only to data necessary for an approved use case, with sensitive information minimized, deidentified or anonymized whenever possible.
AI tools at ChristianaCare also undergo review by an AI governance group that considers security, privacy, compliance, risk and operational impact before use. The health system prohibits sensitive patient, employee, financial or proprietary information from being entered into public or unapproved AI platforms.
But health systems’ scrutiny does not stop at determining which data an AI tool can access. Leaders said they are also placing restrictions on what vendors can do with that information once they have it.
CHOP prohibits vendors from using its data for purposes beyond those for which it was originally shared. In some agreements, the health system allows vendors to use deidentified data to improve capabilities CHOP already uses, but generally does not permit it to be used to develop or improve unrelated capabilities.
Stanford similarly prohibits vendors from using its data to train models or improve products for other customers without explicit consent. Its contracts can also establish where data is stored, how long it can be retained, when it must be deleted and whether vendors can attempt to reidentify deidentified information.
Those requirements extend down the technology supply chain, according to Mr. Lindmark.
Stanford requires applicable data restrictions to extend to subcontractors and subprocessors — an issue he said is becoming more important as AI vendors build products on top of third-party foundation models.
For some health systems, disagreement over those terms can end a potential AI deal.
Mr. Lindmark said Stanford has walked away from vendors over data and contractual requirements and views doing so as “routine rather than a last resort.”
Common sticking points include vendors seeking rights to use Stanford’s data to train products sold to other customers, vague data-use provisions, resistance to audit rights and unwillingness to disclose underlying models or subprocessors, he said.
Liability can also become a breaking point.
“Some vendors cap liability at a fraction of contract value while leaving us exposed if a breach or model failure causes real harm, a mismatch we won’t accept for tools touching clinical workflows or PHI [protected health information],” Mr. Lindmark said.
CHOP has also chosen not to move forward with vendors because of data concerns, according to Mr. Salmasian.
More frequently, he said, vendors have entered negotiations seeking broader access or usage rights, and the health system has negotiated those provisions down to terms it considers acceptable.
As AI becomes more deeply embedded in healthcare operations, Mr. Salmasian said health systems should avoid treating compliance with federal health privacy law as the endpoint of their oversight.
“The healthcare industry needs to broaden the conversation beyond HIPAA compliance,” he said. “HIPAA doesn’t represent all privacy matters, and privacy is only one dimension of the risk.”
One concern is how quickly AI technologies are changing. Mr. Salmasian said development and updates are moving faster than even well-resourced health systems’ ability to evaluate, monitor and implement the tools, making strategic vendor selection increasingly important.
Health systems should also consider the longer-term business implications of sharing their data, he said.
“Across multiple industries, we have seen technology companies use customer data and workflow insights to develop new products that eventually compete with, or displace, parts of the markets they originally served,” Mr. Salmasian said. “Healthcare may not be immune to that dynamic.”
At Stanford, another emerging concern is agentic AI.
Mr. Lindmark said today’s environment remains manageable because relatively few AI agents are operating inside the organization and Stanford uses human review for consequential decisions. But that governance model could become difficult to sustain if health systems eventually have hundreds or thousands of agents operating across clinical and administrative workflows.
“My biggest fear is agentic AI outpacing our ability to monitor it correctly,” he said.
At that scale, health systems will need real-time visibility into what AI agents are doing, mechanisms to detect when they stray from their intended purpose and the ability to intervene quickly, Mr. Lindmark said.
Stanford is already investing in those monitoring capabilities.
“Retrofitting oversight after agents are already embedded at scale would be far riskier than building it in from the start,” he said.
At the Becker's 11th Annual IT + Revenue Cycle Conference: The Future of AI & Digital Health, taking place September 14–17 in Chicago, healthcare executives and digital leaders from across the country will come together to explore how AI, interoperability, cybersecurity, and revenue cycle innovation are transforming care delivery, strengthening financial performance, and driving the next era of digital health. Apply for complimentary registration now.
