# Control for variables: multiple regression vs residuals

I'm trying to understand how to control for variables in linear regression. Say I want to predict $\mathrm{outcome}$ using $x_1$ and control for $x_2$ and $x_3$. The common approach I see is just putting them all in a linear model:

lm(outcome ~ x1 + x2 + x3)


Another approach is to calculate the residuals of $x_1$ and then use them in prediction:

lm(outcome ~ lm(x1 ~ x2 + x3)$residuals)  The coefficient for$x_1$in these methods is the same, but its statistical significance is different. Which approach is better? It seems that from the first we cannot interpret the significance of$x_1$because we have colinearity, is it true? Are there any other things I should notice (assumptions etc) in this analysis? Example of the difference in outcomes: > x2 <- runif(n=100, 0, 10) > x3 <- runif(n=100, 0, 10) > x1 <- 0.3*x2 + 0.6*x3 + 0.1*runif(100, 0, 10) > outcome <- 2 + 0.5*x1 + 0.25*x2 + 0.25*x3 + rnorm(100, 0, 0.5) > summary(lm(outcome ~ x1 + x2 + x3)) Call: lm(formula = outcome ~ x1 + x2 + x3) Residuals: Min 1Q Median 3Q Max -1.38003 -0.27255 0.05079 0.27100 0.88337 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 2.02248 0.15876 12.739 < 2e-16 *** x1 0.41726 0.15909 2.623 0.0101 * x2 0.27063 0.04612 5.867 6.3e-08 *** x3 0.31244 0.09833 3.177 0.0020 ** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Residual standard error: 0.4598 on 96 degrees of freedom Multiple R-squared: 0.9377, Adjusted R-squared: 0.9357 F-statistic: 481.3 on 3 and 96 DF, p-value: < 2.2e-16 > summary(lm(outcome ~ lm(x1 ~ x2 + x3)$residuals))

Call:
lm(formula = outcome ~ lm(x1 ~ x2 + x3)$residuals) Residuals: Min 1Q Median 3Q Max -3.7353 -1.0978 0.1281 1.2922 4.4606 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 6.8806 0.1819 37.835 <2e-16 *** lm(x1 ~ x2 + x3)$residuals   0.4173     0.6292   0.663    0.509
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 1.819 on 98 degrees of freedom
Multiple R-squared:  0.004467,  Adjusted R-squared:  -0.005691
F-statistic: 0.4398 on 1 and 98 DF,  p-value: 0.5088


## 1 Answer

The first method is correct. "Independent variables", "predictor variables" and "control variables" are all treated identically by regression, the difference is in how you interpret the output.

Whether you have collinearity is a separate question that has to be evaluated on its own. I recommend using condition indexes and proportion of variance. If you do have collinearity, there are a number of ways of dealing with it. These have been dealt with here on CrossValidated many times. You can search for those threads.