IBM addresses Watson Health woes by adding regional treatment advice, STAT finds

Shortly after reports emerged detailing numerous shortcomings with IBM's Watson for Oncology platform, the company is modifying the artificial intelligence-powered software, STAT reports.

Here are 10 things to know:

1. IBM initially touted the success of Watson for Oncology for its work with the prestigious Memorial Sloan Kettering Cancer Center in New York City.

2. A July 25 STAT investigation revealed Watson for Oncology, a clinical decision support platform that  spits out cancer treatment advice using AI trained by physicians at MSK, used only hypothetical patient information to train the system, although IBM repeatedly claimed it used real patient data to derive its recommendations. Internal documents reviewed by STAT from a year ago identified "multiple examples of unsafe and incorrect treatment recommendations."

3. On top of these concerns, physicians using the Watson for Oncology software in countries outside the U.S. have complained the software offers irrelevant or inaccurate treatment recommendations based on American bias.

4. A top IBM executive told Watson Health employees at an internal meeting the day STAT published its report that the Watson for Oncology software would start to incorporate localized treatment advice and, for the first time, real patient data, according to a recording of the meeting obtained by STAT. The software will still be based on MSK physicians' advice, but the company will reportedly work to better understand regional differences in treatment preferences and patient populations.

5. Lisa Rometty, IBM Watson Health vice president and general manager for oncology, life sciences and personal health, told employees at the meeting the company's cancer-related products weren't generating as much revenue as the company hoped. "We did forecast that we would not meet the oncology revenue, and we didn't," Ms. Rometty said. "And even though I want nothing more than to meet our targets, I think equally important though is that you have to be predictable."

6. In a statement to STAT, IBM contrasted Ms. Rometty's point, saying: "We've also doubled revenue for Watson for Oncology and Watson Genomics each year since we launched this new business for IBM." The company did not disclose revenue from the two products in its statement, but it posted second quarter revenue of $20 billion, up 4 percent year-over-year, marking the company's the third consecutive quarter of revenue growth after five straight years of declining profits.

7. Highlighting the importance of regional differences in cancer treatment recommendations at the meeting, Ms. Rometty said: "We're really focused, as an example, on understanding what we need to do to ensure that, at the point of care in China, [the software is] where physician needs [it] to be to effectively provide recommendations to them on how to treat their patients in China … That's going to differ from what a physician needs to have at the point of care in the United States."

8. This shift contrasts IBM's previous stance that international physicians sought out advice from Watson for Oncology because of its MSK-backed recommendations. However, it aligns with advice from former Watson Health Deputy Chief Health Officer Andrew Norden, MD, who suggested last year that the "foundation" of the program should use national treatment guidelines.

9. A new version of Watson for Oncology that includes suggestions based on treatment guidelines from the National Comprehensive Cancer Network in the U.S., which are used worldwide, is reportedly in the works, a hospital official in Thailand told STAT. And, IBM plans to add real patient data to the software, but it is unclear how that data will be used to offer advice.

10. Ms. Rometty pointed to IBM's collaboration with Cota, a data-driven healthcare technology company where Dr. Norden now serves as CMO, as evidence of the company's successful cancer work with real patient data. As part of the collaboration, IBM and Cota completed a pilot project at Edison, N.J.-based Hackensack Meridian Health, combining their capabilities. According to Dr. Norden, the pilot "went well" and the companies are "taking the pilot and scaling it up."

Earlier this year, IBM laid off an undisclosed number of employees from its Watson Health division and three acquired companies — Phytel, Truven and Explorys. Those employees told various media outlets of the company's disorganization and mismanagement, which they alleged sparked questions of its success with AI.

For the complete STAT report, click here.

More articles on artificial intelligence:

Harvard, UPenn researchers use AI to predict mortality for cancer patients
WebMD exec: AI is limited in healthcare — for now
NIH to establish AI, machine learning committee

© Copyright ASC COMMUNICATIONS 2019. Interested in LINKING to or REPRINTING this content? View our policies by clicking here.

 

Top 40 Articles from the Past 6 Months