New study shows how using machine learning in healthcare worsens inequality

Predictive models lack the necessary context to interpret the data they are given

Spread the knowledge

a hospital room in gray tones

Photo by Daan Stevens on Unsplash 

Healthcare providers have to make a lot of decisions when it comes to providing the best patient care, and in some cases, algorithms are used to help with this clinical decision-making process. However, the fairness of such tools is not a guarantee.

As a physician assistant, using algorithms to help guide clinical decision-making has been a frequent occurrence. For example, I’ve used a tool called the Pooled Cohort Equations to estimate a patient’s risk of atherosclerotic cardiovascular disease in a 10-year period. If a patient’s risk exceeds a certain cutoff, statin therapy is recommended. But it is imperative that clinicians understand the limitations of such algorithms.

In a new pre-print posted on arXiv, researchers from Stanford University characterized the trade-offs between a predictive model’s fairness and its performance. Using 25 combinations of datasets, clinical outcomes, and demographic attributes (such as race, ethnicity, gender, sex, or age), they set up a series of predictive models. The models included specific fairness criteria that were adjusted to be more or less strict, and they quantified the effect this had on the model’s performance.

The researchers concluded that there were concerning limitations regarding algorithmic fairness in healthcare. Moreover, they noted that constraining a predictive model in order to achieve fairness was “insufficient for, and may actively work against, the goal of promoting health equity.”

This is partly because predictive models lack the context of how systemic factors lead to health disparities, such as how bias and historical inequalities affect the data going into the models. Furthermore, these factors may also play a role in the intervention triggered by the model’s prediction. Due to this complexity, the researchers suggested that it might be necessary “to abstain from algorithm-aided decision making entirely.”

The use of machine learning in healthcare could potentially worsen health disparities. Therefore, it is necessary for researchers developing healthcare-related predictive models to actively engage in participatory design practices to address the biases currently present in healthcare.