Kaiser Permanente and Massachusetts Institute of Technology researchers built AI models that use only routine EHR data to flag patients at higher risk for gestational diabetes, gestational hypertension and preeclampsia as early as 12 weeks into pregnancy, according to a study in Communications Health.
The team trained 49 machine learning models on 442,760 pregnancies cared for by Oakland, Calif.-based Kaiser Permanente’s Northern California region from 2011 to 2021. The models predicted seven complications at six-week intervals. At 12 weeks, they posted AUC scores of 0.80 for gestational diabetes, 0.79 for gestational hypertension and 0.74 for preeclampsia. AUC measures how well a model separates high-risk patients from low-risk ones, and a score of 1.0 is perfect.
Preterm birth, stillbirth, placental abruption and small-for-gestational-age births were harder to predict. Those models scored 0.62 to 0.68 even at 24 weeks.
The early models outperformed existing risk checklists from the National Institutes of Health and the Society for Maternal-Fetal Medicine. The researchers estimated that steering the top 20% of flagged patients into gestational diabetes lifestyle counseling would save about $10.8 million across the study’s test group. Aspirin prophylaxis for preeclampsia was also cost-effective. A cervical-length screening pathway for preterm birth was not.
The models lost accuracy when tested on 6,105 pregnancies from the NIH’s All of Us Research Program. Local fine-tuning narrowed the gap but did not close it. Co-senior author Marzyeh Ghassemi, PhD, an associate professor at MIT in Cambridge, Mass., said AI models must be evaluated and adapted “before introducing them into different health care settings.”
The models also performed worse for Black patients on preeclampsia and gestational hypertension. The researchers said this gap may stem partly from early-pregnancy blood pressure being recorded earlier and more often for white patients.
Co-senior author Yeyi Zhu, PhD, a research scientist at the Kaiser Permanente Division of Research, said in a Nov. 19 news release that the findings show “AI is not a one-size-fits-all solution for maternal and neonatal health.” The team plans to test whether adding ultrasound measurements and clinical notes improves predictions, and whether using the models in practice improves outcomes for patients.