Why does the OLS-intercept not just "de-mean" the residuals of the same model without intercept?

Question

The answer here explains, why the residuals of an OLS-regression have mean zero if an intercept is included.

Problem:

Intuitively, i would assume that including an intercept just "de-means" the residuals of the same regression without intercept. However, this seems not to be right:

set.seed(123)
x <- 1:100
e <- rnorm(100, sd = 10)
y <- x+e

# OLS-regression with intercept
summary(lm(y~x))

Call:
lm(formula = y ~ x)

Residuals:
     Min       1Q   Median       3Q      Max 
-24.5356  -5.5236  -0.3462   6.4850  20.9487 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) -0.36404    1.84287  -0.198    0.844    
x            1.02511    0.03168  32.356   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 9.145 on 98 degrees of freedom
Multiple R-squared:  0.9144,    Adjusted R-squared:  0.9135 
F-statistic:  1047 on 1 and 98 DF,  p-value: < 2.2e-16

# OLS-regression without intercept
reg <- lm(y~x-1)
summary(reg)

Call:
lm(formula = y ~ x - 1)

Residuals:
     Min       1Q   Median       3Q      Max 
-24.5085  -5.6817  -0.3652   6.2934  20.8238 

Coefficients:
  Estimate Std. Error t value Pr(>|t|)    
x  1.01968    0.01565   65.17   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 9.101 on 99 degrees of freedom
Multiple R-squared:  0.9772,    Adjusted R-squared:  0.977 
F-statistic:  4247 on 1 and 99 DF,  p-value: < 2.2e-16

# mean of residuals
mean(reg$residuals)
[1] -0.08965128

We see, that the estimated intercept has a value of -0.36404 (and the residuals have mean zero). The same model without intercept reports a mean for the residuals of -0.08965.

Question:

The intercept does not just "de-mean" the residuals, so what is the relationship between the intercept of an OLS-regression and the residuals of the same model without intercept?

Answer 1

In simple linear regression with intercept, $y = \alpha + \beta x$, the least squares estimates are $$ \hat{\beta}_I = \frac{\sum_ix_iy_i - n\bar{x}\bar{y}}{\sum_ix_i^2 - n\bar{x}^2}, \qquad \hat{\alpha}_I = \bar{y} - \hat{\beta}_I\bar{x}. $$ In simple linear regression without intercept, $y = \beta x$, the least squares estimate is $$ \hat{\beta}_O = \frac{\sum_ix_iy_i}{\sum_ix_i^2}, $$ and so the residual mean is $$ \bar{r} = \frac{1}{n}\sum_iy_i - \hat{\beta}_Ox_i = \bar{y} - \hat{\beta}_O\bar{x}. $$ In the first model, $\hat{\alpha}_I = \bar{y} - \hat{\beta}_I\bar{x}$ while in the second model, $\bar{r} = \bar{y} - \hat{\beta}_O\bar{x}$. But the estimates for the $\beta$ coefficient are different unless the predictor(s) have mean 0. Then $\bar{x} = 0$ and the formula for $\hat{\beta}_I$ simplifies to the formula for $\hat{\beta}_O$.

Note: For clarity I've used the subscript $I$ to indicate the case with intercept and $O$ -- the case without.

And so you can combine the formulas for $\hat{\alpha}_I$ and $\bar{r}$ to derive the relationship: $$ \hat{\alpha}_I = \bar{r} + (\hat{\beta}_O- \hat{\beta}_I)\bar{x} $$

Answer 2

including an intercept just "de-means" the residuals of the same regression without intercept

You're suggesting that the two fits should be parallel.

Just look at a plot:

You're saying that the green line should be parallel to the red line. That doesn't make sense, you can get much closer to the data than that.