Many Amazon Alexa health skills violate company policies

Julie Spitzer - Print  | 

Although Amazon's artificial intelligence-powered voice assistant Alexa has thousands of health and wellness skills — not to mention experts have touted its potential in the industry — a Quartz investigation revealed many of those skills give poor health advice.

Customers can either use Alexa's voice command search function to ask about their health issue, which Alexa then processes to query answers from an Amazon "verified trusted source" such as Wikipedia or WebMD, or they can download one of Alexa's skills —  which function similar to a smartphone app.

While anyone can develop their own Alexa skill and submit it to Amazon, company policy requires health-related skills meet certain criteria, including:

Failure to abide by these policies — namely provide a disclaimer — means Amazon can remove the skill from its library.

However, Quartz reporters Katherine Foley and Youyou Zhou audited 915 health and wellness skills, and identified 65 medical skills without a disclaimer.

When Quartz notified Amazon of its findings July 2, company spokespeople said it "routinely audit[s] Alexa skills, and if they are not compliant, we work quickly to communicate with the developer and take action on behalf of customers," adding that it conducts the audits on an ongoing basis. Those 65 skills were still available as of July 10.

To test how well the skills perform, the reporters then subdivided the 915 health and fitness skills into four categories based on the above Amazon policy:

  1. Healthcare: skills that offer basic information on health or healthcare
  2. Trackers: skills that help monitor health
  3. Guided activities: skills that serve as basic fitness coaches
  4. Diagnostics and treatment: skills that claim to diagnose diseases or provide treatment options

Skills in Quartz's diagnostics and treatment category — which Quartz claimed were clearly offering health advice despite the disclaimers  — frequently picked up on the wrong keywords from user questions, and subsequently offered inaccurate advice. For example, when asked about flu symptoms, one skill diverted to a definition of the bird flu. Other times, skills "rambled on for five minutes, listing out every possible cause," the report states.

Physicians that spoke with Quartz and reviewed the reporters' analysis agreed Alexa skills provide mediocre health advice in the best-case scenarios.

"These answers all sound like they just extract information from Wikipedia (which contains a lot of incorrect information) using very simple 'yes' or 'no' algorithms," Tian Wang, MD, a neurologist at Washington, D.C.-based Georgetown University, told the news outlet via email. "Based on my judgment, these are all bad responses. If this is the best that Silicon Valley can offer, then it will still take a long way for the robotic machine to take over our jobs."

Rumors have been swirling that Amazon is building a health team around Alexa, which is not yet HIPAA-compliant. But since Alexa can't yet parse out actual conversations, it appears she shouldn't be stepping in for human physicians.

To access the full Quartz report, click here.

More articles on health IT:  

Epic founder Judy Faulkner comes in 3rd on Forbes' richest women list
KLAS: 13 precision medicine vendors with high 'mindshare' among providers
16 hospitals, health systems seeking Allscripts, Cerner, Epic, Meditech talent

© Copyright ASC COMMUNICATIONS 2019. Interested in LINKING to or REPRINTING this content? View our policies by clicking here.

To receive the latest hospital and health system business and legal news and analysis from Becker's Hospital Review, sign-up for the free Becker's Hospital Review E-weekly by clicking here.