How to Assess Your Vendor’s Use of AI

As we look to the future of healthcare, artificial intelligence (AI) can impact every aspect from care delivery to pharmaceutical operations to organizational operations, and we expect to see rapid advancements in these technologies over the next few years. AI is readily accessible by most businesses and could make great improvements in the healthcare industry in delivering better patient experiences and outcomes while reducing the cost of care, which is aligned with the “Triple Aim”.

Many healthcare organizations are currently looking at various AI technologies to better support administrative, operational, and clinical workflows. Accordingly, it seems every vendor is promoting AI as a key aspect of their solution offering. It can be difficult to know if a company is truly selling AI or is using the term or concept because of the popularity of the topic. To help you determine who is providing actual AI, here are four questions you should ask a potential vendor and what to look for in their answers.

“Who is on your AI team?”

Make sure to ask about the vendor’s AI team. Who is on it, how big is it, what are their backgrounds, and most importantly, how does the team communicate with customers. AI team members should have experience in areas such as machine learning or applied data science — if the vendor does not have a team dedicated to AI, that can be a red flag. A major consideration is whether you, the customer, are allowed to interact with the AI team. The vendor should be very comfortable with this because AI can’t learn in a vacuum, it needs direct customer feedback in order to evolve over time.

“How do you train your AI?”

Most AIs are first brought to life with a training set of data. It’s important to know what data was originally used to train the vendor’s models. Was the original training set from one customer or several?  How similar is your data to the training set?  You’ll also want to know what the vendor’s strategy is to improve their training set regularly. Ideally, AIs are re-trained on a continuous basis using new data, creating a feedback loop between the AI and the customers benefiting from the solution. This is especially true in the healthcare industry where data changes often.

“What metrics do you use to evaluate your AI?”

The key performance indicators (KPIs) a vendor uses shows how they define success for their AI. Obvious measures are false negatives and false positives rates but there’s a lot of nuance when it comes to market segments. An average false positive rate for all customers might not mean much if your organization is not an average customer. More important KPIs tie back to the value the AI is delivering such as time saved, revenue saved, or some other ROI.  You’ll also want to know whether updated KPIs are shared beyond the AI team to customers. This level of transparency ensures that everyone is working together to enable the AI to continuously improve and solve real problems.

“What bias has been found in your AI?”

Bias is inherent in AI but can be mitigated with continuous research and refinement. If a vendor’s answers check out on the first three questions, it’s very likely that they’ve also had internal conversation about whether their AI has bias. What are they learning about the bias, and what is their plan to reduce it?

We understand the importance of choosing the right technology and know it can be tricky when it comes to AI. This isn’t meant to be an exhaustive list of questions and welcome thoughts that others might have about how we can hold ourselves accountable when it comes to building purposeful and ethical AI.  In the meantime, we hope this list will help you evaluate potential AI vendors to make the most informed choice.

About Protenus

Protenus harnesses the power of artificial intelligence (AI) to provide healthcare organizations with scalable risk-reduction solutions that drive the safest patient outcomes while protecting the reputation of the organizations. We are committed to innovation, determined to reduce risk, and focused on supporting our community of employees, customers, and ultimately, patients. Empowering healthcare to eliminate risk is at the heart of all we do. Learn more at Protenus.com and follow us on Twitter @Protenus.

Copyright © 2024 Becker's Healthcare. All Rights Reserved. Privacy Policy. Cookie Policy. Linking and Reprinting Policy.

 

Featured Whitepapers

Featured Webinars