The massive potential of AI to transform healthcare by speeding diagnosis, developing highly personalized treatments and cutting costs may have blinded regulators, providers and patients to the equally massive dangers associated with the technology, per the report. The sector has also been heavily influenced by the tech industry’s “move fast and break things” mantra, to its detriment.
Among major and widespread challenges still facing AI in healthcare are gender and racial biases in algorithms, a lack of consistency in the technology’s outcomes across hospitals and health systems, and the “black box” quality of AI that prevents users from knowing exactly how an algorithm reached its conclusion, Mildred Cho, PhD, a professor of pediatrics at Stanford (Calif.) University’s Center for Biomedical Ethics, told KHN and Scientific American.
Additionally, as noted by Eric Topol, MD, director and founder of the Scripps Research Translational Institute, only one medical AI system has undergone randomized clinical testing thus far, and no AI products currently sold in the U.S. have been verified by randomized clinical trials.
“Almost none of the [AI] stuff marketed to patients really works,” Ezekiel Emanuel, MD, PhD, professor of medical ethics and health policy at Philadelphia-based University of Pennsylvania’s Perelman School of Medicine, told the news outlets.
To avoid the potentially disastrous consequences of ineffective AI, the FDA will need to implement more rigorous standards for testing the technology and verifying claims. Additionally, those developing AI for use in healthcare will need to take a more cautious approach to commercializing their offerings.
“If ‘failing fast’ means a whole bunch of people will die, I don’t think we want to fail fast,” said Oren Etzioni, PhD, CEO of the Allen Institute for AI. “Nobody is going to be happy, including investors, if people die or are severely hurt.”
More articles on AI:
8 healthcare companies among the top 25 highest-funded AI startups of 2019
NAM report: The success of healthcare AI depends on building consensus
Mayo Clinic partner Eko earns FDA ‘breakthrough device’ designation
