3
$\begingroup$

Up until now, it was my understanding that multiple regression can be used to estimate the influence of an independent variable (IV) on an outcome (DV) while controlling for the influence of other possibly interfering covariates (by entering them as additional predictors; e.g., age, gender, income, etc.). Then, by looking at the coefficient of the IV of interest, I will get an impression of this variable's influence without the additional impact of the covariates (which are kept constant).

In a way, this implies a "partialling out" interpretation of multiple regression, as stated in this textbook on p. 83.

However, today I stumbled upon a recent 2020 paper (published with relatively high impact!), which claims that this is fundamentally wrong. The author claims that you should only control for so-called "confounding variables" in your regression analyses, i.e., variables that affect both the IV and the outcome. The author further explains that previous research on life satisfaction is all wrong, because it made the mistake of using "income" as a covariate in a regression between age (IV) and life satisfaction (outcome).

Could someone help me wrap my head around this? In my psychological understanding, the criticized procedure is absolutely correct: If I want to estimate the sheer influence of age on life satisfaction without the impact of another interfering variable such as money, I should keep the latter constant by adding it as a covariate, no?

Thank you very much for your help.

$\endgroup$
2
  • 2
    $\begingroup$ You need to decide on the specific effect you want to estimate. If you want the total effect of age on life satisfaction, you should not control for any variables potentially caused by age. If you want the direct effect of age on life satisfaction that does not pass through income, then you need to include income in the model. $\endgroup$
    – Noah
    Jan 25, 2021 at 4:27
  • $\begingroup$ Thank you, @Noah, very comprehensible answer, and it helped me to once again understand the relationship between regression and mediation better. :) $\endgroup$
    – JPS
    Jan 25, 2021 at 13:21

2 Answers 2

2
$\begingroup$

The statement : "Regression models should control only ‘confounding’ variables; that is, variables that are causally prior to the dependent variable and the core independent variable of interest." (Bartram, 2020 - abstract) is wrong.

Only controlling for control variables that are correlated prior to the independent variable and the outcome is wrong. Of course, age can not be "caused", but there could be a suppressor or mediating effect, i.e., an effect intervening between IVs and the outcome. Confounding, suppressor and mediating effects are statistically the same, but conceptually different.

   Here an example build in R:
    
    > #hypothetical example
> n=1000000
> age=rnorm(n)
> anxiety=.5*age+sqrt(1-.5^2)*rnorm(n)
> satisfaction = .4*age-.4*anxiety+sqrt(1-(.4^2+.4^2+2*.4*(-.4)*.5))*rnorm(n)
> 
> #correlation
> cov(cbind(age,anxiety,satisfaction))
                   age    anxiety satisfaction
age          0.9989992  0.5001790    0.2000404
anxiety      0.5001790  0.9991424   -0.1977109
satisfaction 0.2000404 -0.1977109    0.9992774
> 
> #regression with both IV
> lm(satisfaction~age+anxiety)

Call:
lm(formula = satisfaction ~ age + anxiety)

Coefficients:
(Intercept)          age      anxiety  
  0.0001752    0.3994308   -0.3978389  

> 
> #regression with only the one correlated to
> lm(satisfaction~age)

Call:
lm(formula = satisfaction ~ age)

Coefficients:
(Intercept)          age  
  3.786e-05    2.002e-01  

Here, if you don't control for anxiety, you have a biased effect of age $=.20$ instead of the real $.40$, this is because, anxiety is actually a suppressor variable (or mediating effect, depending on how you conceptualized your model).

You can read more on this topic here.

I am not familiar on the topic of age and satisfaction. What strikes me, is the lack of concern for age$^2$, which would point at the "U-shaped" relation (if there is one), and that there is a preference for splitting data in two, which is a questionnable practice at best.

$\endgroup$
2
  • $\begingroup$ Thank you so much for your quick reply. That's exactly what I thought: It's a common (and important) practice to also control for suppressor or mediator effects in regression So, my intuition was correct and the cited paper is just...statistical bollocks? How incredible that such confident claims are published in high-level journals then... $\endgroup$
    – JPS
    Jan 21, 2021 at 21:54
  • $\begingroup$ Thank you @JPS. You would be surprised how often it happens actually... $\endgroup$
    – POC
    Jan 21, 2021 at 22:03
0
$\begingroup$

That paper seems to be talking about a really specific case where things are likely to go wrong with model misspecification. If it was true in general that you shouldn't control for non-confounding covariates, then there wouldn't be block designs.

Since this question was asked, an xkcd comic appeared with the setup:

"But if you control for too many variables, your choices will shape the data, and you'll mislead yourself"

I honestly don't get the "shaping the data" part, but it was a popular comic and if I had to guess, it's probably a combination of the concerns:

  1. Wrong model assumptions on the covariate relationship
  2. Controlling for something that doesn't make sense

For 1, the article talks about a "U-shape" relationship between the response and the covariate, which, when using a linear model, is model misspecification. For the other parameters, there's no bias if variables are uncorrelated in a linear regression, and I believe there is bias in a nonlinear model (incl. logistic regression) even if the variables are uncorrelated. Of course with OLS you're getting the best linear approximation to the non-linear case, and I would expect the bias to be mild even with correlation.

For 2, this is where drawing the causal directed graph helps because you can see things that don't make sense to "control for" (in the sense of holding them constant) when you're interested in cause and effect: namely intermediate causes (aka "mediators") and downstream effects from the cause of interest (aka "colliders").

However, I'd point out that putting both colliders and mediators in your model is a great way to increase the passive predictive ability of your model, if that's what you're after.

Even if you're interested in strictly the causal case, the benefits of going after non-confounding covariates is reducing residual error, thus giving you more power to detect treatment effects that you wouldn't be able to otherwise.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.