7
$\begingroup$

I am trying to use DAGs to think more carefully about my regression models. I have a question about the "backdoor criterion", as usually seen in the DAG below (we are interested in the effect of $X$ on $Y$). One way to deal with confounding in this case is to include $Z$ in the regression, i.e.~control for it.

But if $Z$ is something that we can measure - for concreteness, in an economic context, $Z$ might be GDP, which often shows up as potential confounder in my DAGs - is it legitimate to first regress $X$ on $Z$ in order to verify that the hypothesised link $Z \to X$ actually exists? If it does, then we go ahead and include $Z$ in the model; if not, the DAG was incorrectly specified, and $Z$ is not in fact a confounder and does not need to be included.

Is this approach reasonable? What am I missing?

enter image description here

$\endgroup$

2 Answers 2

15
$\begingroup$

Indeed, given the DAG, you should only see a correlation between X and Z if there is a direct link between the two, and thus you could test for a correlation directly. These and similar tests are done by all causal discovery algorithms that automatically create the DAG from data, such as the PC algorithm.

However, from the practical perspective of a multiple regression, there are two caveats to this approach:

a) How do you perform the test? A n.s. p-value is not a proof that effect size is zero. Even if the CI is small and overlaps zero, you cannot exclude small confounding effects and thus you have to trade off a small possible bias (from not including a weak confounder) against the improved precision that you gain by having a model with fewer d.f. Plus, there is the issue of post-selection inference, i.e. you need to correct p-values for the tests performed in the model selection.

b) Second, you say that if a variable is not a confounder, it "does not need to be included". I would add to this: for reasons of bias (= causal perspective). However, there are other reasons to include a variable in a multiple regression. Most importantly, if Z is a strong predictor, i.e. if there is a strong link Z->Y, including Z in the regression can reduce the uncertainty on the estimate X->Y, even in the absence of a confounding link Z->X.

$\endgroup$
9
  • $\begingroup$ Thank you for this excellent, comprehensive response. (I will wait a day before accepting.) $\endgroup$
    – Anthony
    Commented Nov 10, 2022 at 14:38
  • 3
    $\begingroup$ Great answer. See also Pearl (1998) "Why There Is No Statistical Test for Confounding, Why Many Think There Is, and Why They Are Almost Right" $\endgroup$
    – Noah
    Commented Nov 10, 2022 at 15:33
  • 1
    $\begingroup$ @Noah This is a huge assertion - I've always assumed you confirm confounders after drawing the diagram by testing for correlation (p-values & effect size) between both Z -> X and Z -> Y. $\endgroup$
    – RobertF
    Commented Nov 10, 2022 at 18:19
  • 2
    $\begingroup$ @RobertF, see e.g. Glymour et al., 2019 Review of Causal Discovery Methods Based on Graphical Models frontiersin.org/articles/10.3389/fgene.2019.00524/full ... as for any dichotomous decision, they have to trade off type I / II error rates, so of course they will not always generate the correct graph. $\endgroup$ Commented Nov 11, 2022 at 9:07
  • 1
    $\begingroup$ @RobertF - the whole point of these algorithms is to separate correlation from causation. You have to read the literature, but the tldr is: it's possible up to a point! $\endgroup$ Commented Nov 14, 2022 at 10:41
5
$\begingroup$

Yes, in order to confirm a confounding relationship you may perform a regression (or Chi-squared test or other suitable model) of $Z$ on $X$ and $Z$ on $Y$. This is exactly what I'm doing right now for a difference-in-differences healthcare analysis.

There may be confounders or colliders, measured or unmeasured, associated with $Z$ and $X$ or $Z$ and $Y$ that can cloud the true causal relationship between the variables. Therefore, it's important to flesh out the DAG prior to performing regression tests.

$\endgroup$
7
  • $\begingroup$ Thank you, @Robert! Would you happen to have a reference for this "confounder testing"? $\endgroup$
    – Anthony
    Commented Nov 10, 2022 at 13:40
  • 1
    $\begingroup$ Robert, you can test z ~ x but I do not see how you could test z ~ y because there is a mediator path z ~ x ~ y that could cause a correlation in z ~ y even in the absence of a direct link. You would have to condition on X but then you might as well fit the entire DAG. $\endgroup$ Commented Nov 10, 2022 at 13:45
  • $\begingroup$ Broadly speaking, you would be engaging in model selection here, which is a task with nontrivial issues - pretesting effects etc. $\endgroup$ Commented Nov 10, 2022 at 13:56
  • 1
    $\begingroup$ @FlorianHartig Will have to think on this some more - if X has a very weak causal relationship with Y, but Z -> X and Z -> Y are stronger there will be less information "leaking through" to Y. $\endgroup$
    – RobertF
    Commented Nov 10, 2022 at 18:58
  • 1
    $\begingroup$ @RoberF - sure, but in principle, you can see Z ~ Y even if there is no Z -> Y, and the reverse, so it's not a reliable test. $\endgroup$ Commented Nov 14, 2022 at 10:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.