Suppose that you are performing a linear regression examining the main effect $x_1$ and want to adjust for possible confounders $x_2, x_3, x_4$. Is it better to have an unadjusted model and a model adjusted for all potential confounders? Or should you also consider models adjusted for only some of the confounders (e.g. $x_2$, $x_2$ and $x_3$, etc.)?

  • $\begingroup$ I've usually seen main effect compared with all. But I think this might be different based on the area of research. I have also less often seen main effect compared with one type of confounder (say demographics), then another type (say geographic factors) when this was thought to be interesting. $\endgroup$
    – charles
    Mar 10 '14 at 15:30
  • 2
    $\begingroup$ Can you say how you are selecting your confounders? You seem to be focused on a causal question (rather than a purely predictive question) because you are asking about a main "effect" and confounders. If so, you should include potential confounders based on theory, not statistical attributes. If "all potential" confounders is a reasonable number, well thought out, and justified by prior knowledge, then leave them in...if "all potential" confounders is just a long list of covariates in your dataset that haven't been given that much thought, then you have a different problem. $\endgroup$
    – D L Dahly
    Mar 10 '14 at 15:31

I assume you're trying to estimate the causal effect of $x_1$ on $y$, rather than just trying to predict $y$. In general, to find a correct conditioning set (if one exists), you need to know the causal relationships among the variables. Here are some examples to illustrate why:

  1. You must be sure that the variables you're controlling for are actually confounders. If any one of them is not a confounder but instead a common effect, then you must not control for it. For example, say that $x_1$ and $y$ both influence $x_2$; then controlling for $x_2$ will induce an association between $x_1$ and $y$. This association will bias your estimate of the true effect of $x_1$ on $y$.

X2 is a common effect of X1 and Y

  1. If $x_2$ mediates the relationship between $x_1$ and $y$ – that is, if $x_1$ influences $x_2$, and then $x_2$ influences $y$ – then conditioning on $x_2$ will remove this indirect effect of $x_1$ on $y$ from your estimate. If you are interested in only the direct effect of $x_1$ on $y$, then this is the right approach, but if you want to estimate the total effect, then you should not control for $x_2$.

X2 mediates the effect of X1 on Y

  1. In some cases there is no set of variables we can condition on to get an unbiased estimate of the effect. Here is an example: the "M-structure".

U1 influences X1 and X2; U2 influences X2 and Y; X2 influences X1 and Y

In this case, the true effect of $X_1$ on $Y$ is zero. However, there are two unobserved confounders $U_1$ and $U_2$, and one observed confounder $X_2$. If we had observed $U_1$ and $U_2$ we could condition on all three confounders and get an unbiased estimate. However, since we only observed $X_2$ we are stuck. If we don't control for $X_2$ it will confound our estimate of the effet of $X_1$ on $Y$. But if we do control for it, we induce an association between $U_1$ and $U_2$, and therefore between $X_1$ and $Y$.

There are many cases in which you should not control for a particular covariate. If you don't know the causal structure, you may accidentally bias your estimate by controlling for the wrong set of covariates. In this case you can apply a causal structure learning algorithm as a first step, before you try to estimate the causal effect of $x_1$ on $y$.

  • $\begingroup$ Sorry for reviving an old answer, but could you explain, in your first example, how adjusting for X2 would induce a correlation between Y and X1? $\endgroup$
    – Hank Lin
    Jan 14 '19 at 23:06
  • $\begingroup$ @hlinee the phenomenon is sometimes called "explaining away". You can verify it using a simple Bayes net like this example. You can get the intuition behind it from this answer. $\endgroup$ Jan 18 '19 at 0:29

From a causal identification perspective, if $x_2$, $x_3$ and $x_4$ are really confounders then they are by definition common causes (or causes of common causes, etc.) of $x_1$ and $y$. Given a theoretically motivated graph structure that reflects their relationships, you need to control for at least as many of them as is necessary to block backdoor paths from $x_1$ to $y$. You can include more than those variables, but it won't help (or hurt) identification. Here's a succinct review of the issues.

From a prediction perspective you should include as many of them as you find works well, typically according to some out of sample / cross-validated / model-selection measure of model quality.

To combine both perspectives at once you should include enough of them to identify the causal effect of interest, plus any that predict $y$ strongly. The latter will increase the precision of your effect estimate but not affect identification.


Hard to know the level of detail you require but here is a basic answer:

Because of the way you have described your variables as a main effect and confounders I assume you are only really interested in getting a measure of the 'unique effect' of $x$1, controlling for the others, which are of secondary interest. you just want to know how $x$1 this effects your outcome.

There are a number of things to consider:

Conceptually speaking, a model adjusted for all would give you a measure of the unique effect of $x1$ controlling for all the others. (i.e the effect of $x$1 if all other variables were held constant). You want this measure if you are interested in just how X1 on your outcome when its not being mediated by or confounded by other things. What i like to call a more 'pure' effect.

You could run a single model or a new model per new confounder entered and examine $R2$ change. But either way there is the issue of the regression method used (e.g. heirarchical, forced entry or stepwise). This will affect the order that the variables are entered into analysis and their effects in the model. Forced entry may be best but you should read up on these.

Further there is the problem of overfitting you can't keep adding confounders to get a 'purer' measure of $x$1 as your model will not cross-validate well. You can look for the adjusted $R2$ declining with extra predictors to spot this. Cross validation might less important if you aren't seek a perfect predictive model but are more interested in describing your dataset

Hope this gives some pointers

  • $\begingroup$ What happens if the direction of your main effect changes after adding confounders? $\endgroup$ Mar 10 '14 at 15:56
  • $\begingroup$ There was some suppression by your confounders $\endgroup$
    – user41270
    Mar 10 '14 at 20:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.