My team at a health insurance company is discussing how we might measure racial bias in the various predictive models our company uses to assess future health risk (such as annual medical cost or probability of major surgery). This has jumped to the priority following the recently published article in Science (Obermeyer et al., 2019).

One suggestion at assessing bias is to run chi-squared tests on the predicted outcomes among race or ethnicity categories. This seems too simplistic to me - surely we want to control for confounding factors if we want an unbiased estimate of bias(!).

This leads to an unusual proposition of conducting a causal inference on predictive model outcomes in order to estimate the average causal effect of race. The same methodology could then be applied to measure model bias in ethnicity, age, gender, etc.

Question: Is it a valid approach to use a counterfactual model (i.e., multiple regression, exact matching, propensity score matching, etc.) to measure racial bias in predictive model outcomes by using the same data used to build the predictive model?


1 Answer 1


I think this sounds like an interesting and important question. Whether it's a valid approach - I don't know.

Typically, I have always seen causal analyses correspond to some variable that's one is able to externally fix. This is of course not the case with race, gender, etc. However, any variable can have an effect that's obscured by confounders, and you can condition away the confounders as is always done, so it seems like this is reasonable. So, the idea of getting an adjusted effect is reasonable.

But suppose for example you see that the ATE for something like race is 0.5. Does that tell you something about racial bias of the model? Would not a "racially biased model" be so because it gives poor (ie, lots of mistakes) prediction for certain subgroups, but good predictions for others? Would that be detected by measuring the ATE of race?

That it gives simply different predictions for different racial groups (ie, that the ATE is not zero) does not always amount to racial bias. For example, race may be a real risk factor for a certain disease. This would of course be bias if for example the model were used for credit scoring, because we don't want it to take something like race into account. However, for something like assessing risk of disease X, is it bias? Of course if the company is changing premiums based on this information, that might then amount to bias. So, I guess it would be important to assess the approach in the context of the overall goal of the modeling.

This is a very active and important field. These may be relevant:

  • Kusner MJ, Loftus J, Russell C, Silva R. Counterfactual fairness. InAdvances in neural information processing systems 2017 (pp. 4066-4076).

  • Kilbertus N, Carulla MR, Parascandolo G, Hardt M, Janzing D, Schölkopf B. Avoiding discrimination through causal reasoning. InAdvances in neural information processing systems 2017 (pp. 656-666).

I also asked a question that might be relevant some time ago.

  • $\begingroup$ You raise several good points. If predictions of, say, future medical cost for African-Americans are much less precise than for Whites, but not biased, the ATE for race would be close to zero & not statistically significant. You'd need to conduct a separate test that compares the variances of the predictions in the two groups, such as an F-test. $\endgroup$
    – RobertF
    Dec 31, 2020 at 21:40
  • $\begingroup$ Of course the obvious answer proposed by Obermeyer et al. is not to use biased measures like medical cost as a proxy for predicting future risk, say for diabetes or high blood pressure. In these cases it's tempting to simply add a race variable to the model, but this runs into ethical issues even if we did have self-reported data on insurance member's race. Perhaps a better proxy for race would be to ask members enrolling for insurance if they trust their doctors or hospitals and if they're reluctant to see the doctor because of discrimination. Then use that data in the predictive models. $\endgroup$
    – RobertF
    Dec 31, 2020 at 21:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.