Artificial intelligence holds great promise for improving human health by helping doctors make accurate diagnoses and treatment decisions. It can also lead to discrimination that can harm minorities, women, and economically disadvantaged people.

The question is, when healthcare algorithms discriminate, what recourse do people have?

A prominent example of this kind of discrimination is an algorithm used to refer chronically ill patients to programs that care for high-risk patients. A study in 2019 found that the algorithm favored whites over sicker African Americans in selecting patients for these beneficial services. This is because it used past medical expenditures as a proxy for medical needs.

Poverty and difficulty accessing healthcare often prevent African Americans from spending as much money on healthcare as others. The algorithm misinterpreted their low spending as indicating they were healthy, and deprived them of critically needed support.

 …