Let's assume that we have a well defined directed acrylic graph (DAG) showing correct causal relationships between variables.

And this DAG tells us that we need to adjust for 7 variables to analyse the direct effect of an exposure on an outcome.

We are not interested in finding the best prediction. We would like to know does the exposure cause changes in our outcome. Thus, should we worry about finding the best model, AIC values etc? Would it be enough to check that conditional effects are just reasonable numbers?


2 Answers 2


DAGs are non-parametric. If we assume that all variables in your problem are dichotomous (or categorical) and all assumptions required for valid causal inference are met, you can unbiasedly estimate the causal effect with a saturated model for the outcome adjusting for the 7 variables (alternatively, using inverse probability weighting, or both). Otherwise, and in common applications with limited data, you usually have to make some parametric assumptions to specify a parsimonious model, which can lead to bias due to model misspecification. Fit on observed data is not typically used to guide specification of structural models, although parsimonious models are often preferred for interpretability and reducing model variance.

When using machine learning methods for causal estimation, both regularization and overfitting can result in bias. Naive application of ML methods (i.e. that used for prediction) to estimate causal effects is generally invalid; they can not model individual-level causal effects out of the box. However, there are methods for semi-parametric estimation of causal effects where machine learning methods can be used to model high-dimensional confounders while still modelling the treatment effects parametrically for interpretability.


If you're not interested in prediction, the most important is to guarantee that you have the right model and all the relevant variables to te problem - which is different than "check that conditional effects are just reasonable numbers", you can't know from guess what would be a reasonable number. The overfitting problem (larger AIC) acts more on giving your estimators more variance, which isn't all that bad if you're not looking into prediction, as it won't change consistency of the estimators.

EDIT: As Richard pointed out, it's not as simple as I made it look like. Whether it's for prediction or explanation, it's always good to find parsimonious models. As i said, your main goal is to include every variable in the true model, but as you can't know for sure which those are, it might be good to not include the ones you don't have much justification in the name of smaller variance, because even if you have consistent estimators (and can talk about causality), large variance means that your estimates will be largely affected by noise on the sample.

  • $\begingroup$ Overfitting will make the estimated causal effects potentially very different from the actual values, so it is a problem. $\endgroup$ Commented Oct 26, 2020 at 13:20
  • $\begingroup$ Yes it is, but in his case is less of a problem than not including a relevant variable, so the AIC shouldn't be the first method of choosing the variables for him, right? $\endgroup$ Commented Oct 26, 2020 at 13:23
  • $\begingroup$ I am not entirely sure. See e.g. "Paradox in model selection (AIC, BIC, to explain or to predict?)". According to the paper cited there, BIC (which omits more variables than AIC) is appropriate for explanation, and explanation means causal explanation in that paper. I think model selection is a tough question. $\endgroup$ Commented Oct 26, 2020 at 14:08
  • $\begingroup$ True, it is a tough question, check my edit and say what you think. $\endgroup$ Commented Oct 26, 2020 at 14:30
  • 1
    $\begingroup$ I think it looks better now. $\endgroup$ Commented Oct 26, 2020 at 14:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.