Using counterfactual modeling techniques to assess racial bias in predictive models

Question

My team at a health insurance company is discussing how we might measure racial bias in the various predictive models our company uses to assess future health risk (such as annual medical cost or probability of major surgery). This has jumped to the priority following the recently published article in Science (Obermeyer et al., 2019).

One suggestion at assessing bias is to run chi-squared tests on the predicted outcomes among race or ethnicity categories. This seems too simplistic to me - surely we want to control for confounding factors if we want an unbiased estimate of bias(!).

This leads to an unusual proposition of conducting a causal inference on predictive model outcomes in order to estimate the average causal effect of race. The same methodology could then be applied to measure model bias in ethnicity, age, gender, etc.

Question: Is it a valid approach to use a counterfactual model (i.e., multiple regression, exact matching, propensity score matching, etc.) to measure racial bias in predictive model outcomes by using the same data used to build the predictive model?

Answer 1

I think this sounds like an interesting and important question. Whether it's a valid approach - I don't know.

Typically, I have always seen causal analyses correspond to some variable that's one is able to externally fix. This is of course not the case with race, gender, etc. However, any variable can have an effect that's obscured by confounders, and you can condition away the confounders as is always done, so it seems like this is reasonable. So, the idea of getting an adjusted effect is reasonable.

But suppose for example you see that the ATE for something like race is 0.5. Does that tell you something about racial bias of the model? Would not a "racially biased model" be so because it gives poor (ie, lots of mistakes) prediction for certain subgroups, but good predictions for others? Would that be detected by measuring the ATE of race?

That it gives simply different predictions for different racial groups (ie, that the ATE is not zero) does not always amount to racial bias. For example, race may be a real risk factor for a certain disease. This would of course be bias if for example the model were used for credit scoring, because we don't want it to take something like race into account. However, for something like assessing risk of disease X, is it bias? Of course if the company is changing premiums based on this information, that might then amount to bias. So, I guess it would be important to assess the approach in the context of the overall goal of the modeling.

This is a very active and important field. These may be relevant:

• Kusner MJ, Loftus J, Russell C, Silva R. Counterfactual fairness. InAdvances in neural information processing systems 2017 (pp. 4066-4076).

• Kilbertus N, Carulla MR, Parascandolo G, Hardt M, Janzing D, Schölkopf B. Avoiding discrimination through causal reasoning. InAdvances in neural information processing systems 2017 (pp. 656-666).

I also asked a question that might be relevant some time ago.