# Why does the OLS-intercept not just “de-mean” the residuals of the same model without intercept?

The answer here explains, why the residuals of an OLS-regression have mean zero if an intercept is included.

## Problem:

Intuitively, i would assume that including an intercept just "de-means" the residuals of the same regression without intercept. However, this seems not to be right:

set.seed(123)
x <- 1:100
e <- rnorm(100, sd = 10)
y <- x+e

# OLS-regression with intercept
summary(lm(y~x))

Call:
lm(formula = y ~ x)

Residuals:
Min       1Q   Median       3Q      Max
-24.5356  -5.5236  -0.3462   6.4850  20.9487

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -0.36404    1.84287  -0.198    0.844
x            1.02511    0.03168  32.356   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 9.145 on 98 degrees of freedom
Multiple R-squared:  0.9144,    Adjusted R-squared:  0.9135
F-statistic:  1047 on 1 and 98 DF,  p-value: < 2.2e-16

# OLS-regression without intercept
reg <- lm(y~x-1)
summary(reg)

Call:
lm(formula = y ~ x - 1)

Residuals:
Min       1Q   Median       3Q      Max
-24.5085  -5.6817  -0.3652   6.2934  20.8238

Coefficients:
Estimate Std. Error t value Pr(>|t|)
x  1.01968    0.01565   65.17   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 9.101 on 99 degrees of freedom
Multiple R-squared:  0.9772,    Adjusted R-squared:  0.977
F-statistic:  4247 on 1 and 99 DF,  p-value: < 2.2e-16

# mean of residuals
mean(reg$residuals) [1] -0.08965128  We see, that the estimated intercept has a value of -0.36404 (and the residuals have mean zero). The same model without intercept reports a mean for the residuals of -0.08965. ## Question: The intercept does not just "de-mean" the residuals, so what is the relationship between the intercept of an OLS-regression and the residuals of the same model without intercept? ## 2 Answers In simple linear regression with intercept, $$y = \alpha + \beta x$$, the least squares estimates are $$\hat{\beta}_I = \frac{\sum_ix_iy_i - n\bar{x}\bar{y}}{\sum_ix_i^2 - n\bar{x}^2}, \qquad \hat{\alpha}_I = \bar{y} - \hat{\beta}_I\bar{x}.$$ In simple linear regression without intercept, $$y = \beta x$$, the least squares estimate is $$\hat{\beta}_O = \frac{\sum_ix_iy_i}{\sum_ix_i^2},$$ and so the residual mean is $$\bar{r} = \frac{1}{n}\sum_iy_i - \hat{\beta}_Ox_i = \bar{y} - \hat{\beta}_O\bar{x}.$$ In the first model, $$\hat{\alpha}_I = \bar{y} - \hat{\beta}_I\bar{x}$$ while in the second model, $$\bar{r} = \bar{y} - \hat{\beta}_O\bar{x}$$. But the estimates for the $$\beta$$ coefficient are different unless the predictor(s) have mean 0. Then $$\bar{x} = 0$$ and the formula for $$\hat{\beta}_I$$ simplifies to the formula for $$\hat{\beta}_O$$. Note: For clarity I've used the subscript $$I$$ to indicate the case with intercept and $$O$$ -- the case without. And so you can combine the formulas for $$\hat{\alpha}_I$$ and $$\bar{r}$$ to derive the relationship: $$\hat{\alpha}_I = \bar{r} + (\hat{\beta}_O- \hat{\beta}_I)\bar{x}$$ • Thanks for the helpful explanation, but furthermore i would be interested in a more detailed description of the relationship between$\hat{\alpha}$and$\bar{r}$, so does there exist a functional relation between those two parameters? – skoestlmeier Mar 5 '19 at 9:43 • Do you have in mind a particular case when there is a meaningful choice between including the intercept or not? What do you mean by a functional relationship? – dipetkov Mar 5 '19 at 22:21 • No, my question arises in general with no specific case in mind (i am also aware of this discussion about the intercept). With functional relationship i mean if there is defined a mapping between$\hat{\alpha}_I$and$\bar{r}$, i.e. a function$f(\cdot)$where$\hat{\alpha}_I=f(\bar{r})$holds. – skoestlmeier Mar 8 '19 at 9:51 • Sure you can by adding and subtracting a$\hat{\beta}\bar{x}\$ term appropriately. – dipetkov Mar 8 '19 at 11:19

including an intercept just "de-means" the residuals of the same regression without intercept

You're suggesting that the two fits should be parallel.

Just look at a plot:

You're saying that the green line should be parallel to the red line. That doesn't make sense, you can get much closer to the data than that.

• Sometimes, a simple picture says more than thousands words (+1)! Also as commented on the other answer, is there a way to calculate the intercept of the regression, given the residuals of the regression without intercept? – skoestlmeier Mar 8 '19 at 9:57