Dr. Eric Topol turns critical eye on landmark FDA-approved AI device

In April, the FDA approved the first medical device to provide physicians with autonomous screening decisions. It's a milestone for the medical community  — but a few "blind spots" may have been overlooked, two researchers write in an editorial for npj Digital Medicine.

The software, IDx-DR, applies an artificial intelligence algorithm to images of a patient's eye to detect diabetic retinopathy, a diabetes-related eye disease. Since IDx-DR offers its decision without requiring a specialist to interpret the results, providers who aren't typically involved in eye care — such as primary care physicians — are able to use the device for routine screening.

To evaluate IDx-DR, the FDA reviewed data from a clinical study of 900 patients with diabetes at 10 primary care sites, published in npj Digital Medicine. In an associated editorial, Pearse A. Keane, MD, an ophthalmology researcher at Moorfields Eye Hospital in London, and Eric J. Topol, MD, director of the Scripps Translational Science Institute in La Jolla, Calif., commented on the study's shortcomings, while also acknowledging its work as an "an important clinical milestone."

Dr. Keane is also a consultant ophthalmologist at Moorfields, a hospital working on a research project with DeepMind — a sister company of Google and Alphabet's AI arm — to develop an AI system for eye care. Dr. Topol is on the advisory board of Verily Life Sciences, Alphabet's life sciences arm.

"While it is always easy to be critical of studies that forge new ground, it is important to applaud the authors for this pivotal work," they wrote.

Here are three potential issues Dr. Keane and Dr. Topol outlined in their editorial:

1. Will the algorithm scale for the entire population? The two researchers point out that the study only recruited patients from 10 primary care sites — a relatively small sample size, given IDx-DR's target market is the general population. And since only a small proportion of the 900 patients considered in the study had severe forms of diabetic retinopathy, it's still unclear whether the system is effective for these cases.

There's a "gulf between developing a scientifically sound algorithm and its use in any meaningful real world applications," the authors write. "It is one thing to develop an algorithm that works well on a small dataset from a specific population, it is quite another to develop one that will generalize to other populations and across different imaging modalities."

2. Do patients benefit from this technology? In 1998, the FDA approved computer-aided detection systems for mammography. Twenty years later, a study found these systems don't improve diagnostic accuracy of mammography — and may actually miss cancers. To avoid issues like these in the future, Dr. Keane and Dr. Topol stressed the importance of asking the question, "Do patients ultimately have good — or at least noninferior — visual outcomes when [IDx-DR] is used?" earlier, rather than later.

3. Will clinics adopt the software? The University of Iowa Health Care in Iowa City deployed IDx-DR in one of its diabetes and endocrinology centers in June, marking the first U.S. health system to implement the software. It remains to be seen what types of facilities will implement the software going forward, and with what degree of frequency.

"Will primary care clinics incorporate retinal screening into their practice?" the authors ask. "This is not really an 'autonomous system' since someone needs to acquire the image. Who will perform that?"

To read Dr. Keane and Dr. Topol's editorial, click here.

Editor's note: This article was updated Sept. 5 to reflect Dr. Keane and Dr. Topol's affiliation with Alphabet projects.

More articles on artificial intelligence:
NYU School of Medicine teams up with Facebook to improve MRIs
Director of analytics tops list of top-paying AI jobs
Indiana U School of Medicine partners with Fujifilm for AI research

© Copyright ASC COMMUNICATIONS 2021. Interested in LINKING to or REPRINTING this content? View our policies by clicking here.


Featured Whitepapers

Featured Webinars