I'm trying to understand the proof at https://en.wikipedia.org/wiki/Partition_of_sums_of_squares

However, I'm stuck on the line

$$\sum_{i=1}^n (\hat{y}_i - \bar{y})^2 + \sum_{i=1}^n \hat{\epsilon}_i^2 + 2 ( \beta_0 - \bar{y} ) \sum_{i=1}^n \hat{\epsilon}_i + 2 \hat{\beta}_1 \sum_{i=1}^n \hat{\epsilon}_i x_{i1} + \cdots + 2 \hat{\beta}_p \sum_{i=1}^n \hat{\epsilon}_i x_{ip} $$

and all those $\epsilon$ sums go to 0.

The explanation is that

The requirement that the model includes a constant or equivalently that the design matrix contains a column of ones ensures that $\displaystyle \sum_{i = 1}^n \hat{ε}_i = 0 $.

What does this mean? What is this constant? Is it the intercept? Why would that cause the sums to go to 0?

Since $\epsilon_i = y_i - \hat{y}_i$, the difference between observed and fitted Ys, I can understand that $\sum_{i=1}^n \epsilon_i$ might sum to 0 if $n \to \infty$, but why would this be true in general. Just because the sum of squares are minimized doesn't mean the residuals are balanced above and below the fitted line. Or does it?

  • $\begingroup$ It is not the sum but the average of the $\epsilon$s that should equal 0 in the limit. $\endgroup$ – Michael Chernick Jan 27 '18 at 21:09
  • $\begingroup$ It's not asymptotics, it's just algebra. This answers your question. $\endgroup$ – A. G. Jan 29 '18 at 18:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.