Control for variables: multiple regression vs residuals

Question

I'm trying to understand how to control for variables in linear regression. Say I want to predict $\mathrm{outcome}$ using $x_1$ and control for $x_2$ and $x_3$. The common approach I see is just putting them all in a linear model:

lm(outcome ~ x1 + x2 + x3)

Another approach is to calculate the residuals of $x_1$ and then use them in prediction:

lm(outcome ~ lm(x1 ~ x2 + x3)$residuals)

The coefficient for $x_1$ in these methods is the same, but its statistical significance is different.

Which approach is better? It seems that from the first we cannot interpret the significance of $x_1$ because we have colinearity, is it true? Are there any other things I should notice (assumptions etc) in this analysis?

---

Example of the difference in outcomes:

> x2 <- runif(n=100, 0, 10)
> x3 <- runif(n=100, 0, 10)
> x1 <- 0.3*x2 + 0.6*x3 + 0.1*runif(100, 0, 10) 
> outcome <- 2 + 0.5*x1 + 0.25*x2 + 0.25*x3 + rnorm(100, 0, 0.5)

> summary(lm(outcome ~ x1 + x2 + x3))

Call:
lm(formula = outcome ~ x1 + x2 + x3)

Residuals:
     Min       1Q   Median       3Q      Max 
-1.38003 -0.27255  0.05079  0.27100  0.88337 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  2.02248    0.15876  12.739  < 2e-16 ***
x1           0.41726    0.15909   2.623   0.0101 *  
x2           0.27063    0.04612   5.867  6.3e-08 ***
x3           0.31244    0.09833   3.177   0.0020 ** 
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.4598 on 96 degrees of freedom
Multiple R-squared:  0.9377,    Adjusted R-squared:  0.9357 
F-statistic: 481.3 on 3 and 96 DF,  p-value: < 2.2e-16

> summary(lm(outcome ~ lm(x1 ~ x2 + x3)$residuals))

Call:
lm(formula = outcome ~ lm(x1 ~ x2 + x3)$residuals)

Residuals:
    Min      1Q  Median      3Q     Max 
-3.7353 -1.0978  0.1281  1.2922  4.4606 

Coefficients:
                           Estimate Std. Error t value Pr(>|t|)    
(Intercept)                  6.8806     0.1819  37.835   <2e-16 ***
lm(x1 ~ x2 + x3)$residuals   0.4173     0.6292   0.663    0.509    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 1.819 on 98 degrees of freedom
Multiple R-squared:  0.004467,  Adjusted R-squared:  -0.005691 
F-statistic: 0.4398 on 1 and 98 DF,  p-value: 0.5088

Answer 1

The first method is correct. "Independent variables", "predictor variables" and "control variables" are all treated identically by regression, the difference is in how you interpret the output.

Whether you have collinearity is a separate question that has to be evaluated on its own. I recommend using condition indexes and proportion of variance. If you do have collinearity, there are a number of ways of dealing with it. These have been dealt with here on CrossValidated many times. You can search for those threads.