2
$\begingroup$

Let's assume that we have a well defined directed acrylic graph (DAG) showing correct causal relationships between variables.

And this DAG tells us that we need to adjust for 7 variables to analyse the direct effect of an exposure on an outcome.

We are not interested in finding the best prediction. We would like to know does the exposure cause changes in our outcome. Thus, should we worry about finding the best model, AIC values etc? Would it be enough to check that conditional effects are just reasonable numbers?

$\endgroup$
1
$\begingroup$

If you're not interested in prediction, the most important is to guarantee that you have the right model and all the relevant variables to te problem - which is different than "check that conditional effects are just reasonable numbers", you can't know from guess what would be a reasonable number. The overfitting problem (larger AIC) acts more on giving your estimators more variance, which isn't all that bad if you're not looking into prediction, as it won't change consistency of the estimators.

EDIT: As Richard pointed out, it's not as simple as I made it look like. Whether it's for prediction or explanation, it's always good to find parsimonious models. As i said, your main goal is to include every variable in the true model, but as you can't know for sure which those are, it might be good to not include the ones you don't have much justification in the name of smaller variance, because even if you have consistent estimators (and can talk about causality), large variance means that your estimates will be largely affected by noise on the sample.

$\endgroup$
5
  • $\begingroup$ Overfitting will make the estimated causal effects potentially very different from the actual values, so it is a problem. $\endgroup$ – Richard Hardy Oct 26 '20 at 13:20
  • $\begingroup$ Yes it is, but in his case is less of a problem than not including a relevant variable, so the AIC shouldn't be the first method of choosing the variables for him, right? $\endgroup$ – Ricardo Semião e Castro Oct 26 '20 at 13:23
  • $\begingroup$ I am not entirely sure. See e.g. "Paradox in model selection (AIC, BIC, to explain or to predict?)". According to the paper cited there, BIC (which omits more variables than AIC) is appropriate for explanation, and explanation means causal explanation in that paper. I think model selection is a tough question. $\endgroup$ – Richard Hardy Oct 26 '20 at 14:08
  • $\begingroup$ True, it is a tough question, check my edit and say what you think. $\endgroup$ – Ricardo Semião e Castro Oct 26 '20 at 14:30
  • 1
    $\begingroup$ I think it looks better now. $\endgroup$ – Richard Hardy Oct 26 '20 at 14:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.