Edison, N.J.-based Hackensack Meridian Health CIO Kash Patel told Becker's that the health system has created a process to ensure that its artificial intelligence algorithms are equitable.
"We've set up a very formal governance group where we have ethics, compliance and legal in the room. So everything I said has gone through that group multiple times. The math is one thing, science is one thing but how it is adopted, interpreted, safety, access, fairness, all of those things are debated."
The health system has partnered with Google to create AI algorithms that can be used for both clinical decision-making and to reduce administrative workload. The health system is working on developing algorithms for safety events, early delirium intervention, capacity planning and palliative care.
Mr. Patel said the health system is working to formalize equitable concerns in all of its AI algorithms. Hackensack Meridian keeps full-time bioethicists on staff and brings them into the AI algorithm creation process, according to Mr. Patel.
"We are trying to be respectful of different voices. Every time we create an algorithm, there are questions that the team will ask us around the underserved community. Is there a bias that I'll be going off of commercial insurance people? Are we making sure we don't take into account the type of insurance they have or what type of socio-economic factors they have into the model?"
Google Health officials have expressed similar concerns that without proper training, healthcare AI models could replicate current healthcare disparities.
"There's going to be an opportunity for people to have even better access to services, [and] to great quality services," Google Health Chief Health Officer Karen DeSalvo, MD, told The Guardian. "But we're a ways to get there. We have a lot of things to work out to make sure the models are constrained appropriately, that they're factual and consistent, and that they follow these ethical and equity approaches that we want to take – but I'm super excited about the potential, even as a doc."