California Attorney General Rob Bonta is urging federal health officials to reconsider a proposed rule that would eliminate certain oversight requirements for AI tools used in healthcare.
In a letter to the Department of Health and Human Services, Mr. Bonta opposed a proposal that would remove federal certification criteria requiring “model cards” for AI-enabled healthcare products. The proposed rule, titled “Health Data, Technology, and Interoperability: ASTP/ONC Deregulatory Actions To Unleash Prosperity,” would scale back regulations tied to transparency and oversight of healthcare technology.
“I oppose the Trump Administration’s proposed rollback of regulations that require clarity about how AI tools used in healthcare were developed and tested. Delivering safe, effective and equitable access to healthcare services must be at the forefront of any attempt to integrate AI and healthcare,” Mr. Bonta said in a March 2 news release.
Model cards are designed to provide information about how AI systems are developed and tested, including potential risks to patients. They are intended to give healthcare providers and regulators insight into how algorithms function and whether they have been evaluated for fairness and bias.
Mr. Bonta said the change would remove “one of the most significant guardrails currently in place on a federal level” governing AI use in healthcare.
AI tools are increasingly used by healthcare providers to inform decisions such as specialist referrals, disease screening and assessment of infection severity. In the news release, Mr. Bonta warned that errors in these systems can have serious consequences for patients and said transparency about how such tools are built and tested is critical to ensuring safe and equitable care.
The model card requirement was introduced during the Biden administration in response to the growing use of automated decision-making systems trained on electronic health records. Under that rule, developers seeking federal certification for healthcare software must disclose information about the data used to train their algorithms and whether those systems have been evaluated for fairness and bias.
The news release cites concerns that algorithms trained on limited datasets may perpetuate existing disparities. It also references a 2019 study that found a widely used hospital algorithm designed to identify high-risk patients exhibited racial bias.
In his letter, Mr. Bonta argued that eliminating the model card requirement would make it more difficult for providers to comply with federal and state nondiscrimination laws. The Affordable Care Act prohibits providers from discriminating based on protected status. According to the news release, Mr. Bonta last year issued guidance to healthcare entities on how California law applies to AI systems used in care delivery.
Without model cards, providers would lose a key tool used to assess whether AI systems meet legal and ethical standards, the news release said.
Mr. Bonta called on federal officials to reverse the proposed rollback and maintain requirements he said help ensure AI technologies in healthcare are safe, effective and equitable.
At the Becker's 11th Annual IT + Revenue Cycle Conference: The Future of AI & Digital Health, taking place September 14–17 in Chicago, healthcare executives and digital leaders from across the country will come together to explore how AI, interoperability, cybersecurity, and revenue cycle innovation are transforming care delivery, strengthening financial performance, and driving the next era of digital health. Apply for complimentary registration now.
