$y_i$ denotes the observed values of $y$, $\hat{y}_i$ are the projected values from the explanatory variables, and $\bar{y}$ is the mean. This relation is supposed to hold up when one of the explanatory variables is the constant. So far, I have gotten $$ \sum (y_i - \bar{y})^2 - \sum (y_i - \hat{y}_i)^2 = \sum(-y_i \bar{y} + 2 y_i \hat{y}_i - \hat{y}_i^2) $$ It seems that if I can somehow show that $\sum y_i \bar{y} = \sum y_i \hat{y}_i$ then the derivation would be complete. Can someone show me how to proceed from here on?

  • $\begingroup$ It is qualified by stating that this is true only when $\hat{\beta}_0 \neq 0$ so the constant explanatory variable contributes to the regression. I agree that this is not true in the general case. $\endgroup$ – Astaboom May 6 '16 at 8:53

Somewhat ugly answer: $$ \begin{align*} \sum_i \left[ \left( y_i - \bar{y}\right)^2 - \left( y_i - \hat{y}_i\right)^2 - \left(\bar{y} - \hat{y}_i \right)^2 \right] &= \sum_i \left( y_i^2 - 2\bar{y}y_i + \bar{y}^2 - y_i^2 + 2y_i \hat{y}_i - \hat{y}_i^2 - \bar{y}^2 + 2\bar{y}\hat{y}_i - \hat{y}_i^2\right) \\ &= \sum_i \left( - 2\bar{y}y_i + 2y_i \hat{y}_i + 2\bar{y}\hat{y}_i- 2\hat{y}_i^2 \right)\\ &= -2n\bar{y}^2 + 2n\bar{y}^2 + 2\sum_i \hat{y}_i\left(y_i - \hat{y}_i \right) \\ &= 2 \sum_i \hat{y}_i\left( y_i - \hat{y_i}\right) \end{align*} $$ Which equals zero because it's basically the orthogonality condition that's used to estimate your $\hat{b}$ vector in ordinary least squares.

Sketch of linear algebra to show this explicitly (it's so much easier and succinct to use matrix notation...)

$$\begin{align*} \sum_i \hat{y}_i\left( y_i - \hat{y_i}\right) &=(Xb)'(y - Xb)\\ &= b'X'y-b'X'Xb \\ &= y'X(X'X)^{-1}X'y - y'X(X'X)^{-1}X'X(X'X)^{-1}X'y\\ &= y'X(X'X)^{-1}X'y - y'X(X'X)^{-1}X'y\\ &= 0 \end{align*} $$

  • $\begingroup$ Thanks for the input. I would have never thought of flipping the LHS to the RHS and equate the whole thing to 0. Nice. Though, just one minor point: when you wrote $\sum (-2 \bar{y} y_i + 2 \bar{y} \hat{y}_i) = -2\bar{y}^2 + 2 \bar{y}$, you actually meant $-2n\bar{y}^2 + 2 n \bar{y}$, right? After all, I do see that $\sum y_i = \sum \hat{y}_i = n \bar{y}$. $\endgroup$ – Astaboom May 7 '16 at 4:56
  • $\begingroup$ And going by the matrix form does make everything simpler by a mile. It's uncanny how simple it becomes. $\endgroup$ – Astaboom May 7 '16 at 4:59
  • 1
    $\begingroup$ @Astaboom good catch, yeah there should be an $n$ there (i just fixed it). $\endgroup$ – Matthew Gunn May 7 '16 at 13:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.