Identifying confounders in multiple linear regression

Question

I am currently trying to identify confounders in a multiple linear regression, but I am a little unsure of a couple of steps. These are the steps I am taking:

1. Check to see if the potential confounders have a significant association with the dependent variable y.
2. Check to see if the potential confounders have a significant association with the main independent variable x.
3. If 1) and 2) are significant, check for a 10% change in beta coefficient.

However, when checking 1) and 2), do I put all of my potential confounders in the model, or do I check them one-by-one? So for example, y = x + gender + education + age, or y = x + gender and then y = x + education and then y = x + age ? My thinking was that when you check them one-by-one, there could be residual variables that are uncontrolled for, whereas if you put them all in one model, then you're looking more at the individual effects of the potential confounder on the dependent and independent variables.

Thanks for the help!

Answer 1

By jumping straight to statistical testing for correlation, you are putting the cart before the horse. You should start by asking, is there any reason, a priori, to expect the putative confounders to have a causal relationship with the 'x' and 'y' variables? Statistical testing alone can't tell you this. Not only that, statistical testing can lead you astray. Consider this set of relationships:

In this case C will show a correlation with both x and y, possibly "significant", but it is not a confounder. If your goal is to get the total causal effect of x on y, it would be a mistake to control for C in this case.

Now consider this set of relationships:

In this case, C is a confounder, and you should control for it. However, depending on the strength of the causal effects and the size and composition of your dataset, the correlations between C and x and y may or may not be "significant". Failing to control for C because one of those tests doesn't meet some arbitrary significance threshold would be a mistake.

To summarize, you should always start by drawing out your hypothesized causal relationships between the relevant variables. Then you can decide what statistical tests are necessary to test your hypothesis.

For more information about this kind of modeling, I recommend Causal Inference in Statistics, by Judea Pearl, Madelyn Glymour, and Nicholas P. Jewell. The topics it covers include how and why to draw graphs like the ones above, and once you have the graphs, how to use them to decide what statistical calculations to perform.

Answer 2

Data driven methods for identifying confounders are fraught with difficulties. As an example, steps 1 and 2 depend on the power of the associated test, and introduce uncertainty into the analysis which can not be properly accounted for in the final inference.

The best approach to identifying confounders is to draw a directed acyclic graph (DAG) for your causal model and determine an identifiability strategy. More on this can be found in most any causal inference text, but I recommend this book and this book.

Additionally, you should never check which variables to include/exclude from the model (for any reason) via a stepwise procedure as you describe. See this answer for more.