The sum of squares decomposition produces this bit of algebra:

$$\sum (y_i - \bar{y})^2 = \sum(y_i - \hat{y}_i)^2 + \sum(\hat{y}_i - \bar{y})^2 + 2 \sum (y_i - \hat{y}_i)(\hat{y}_i - \bar{y}).$$

The "cross term" at the end is zero. I've seen mathematical proofs of this (see this question), but I have no intuition for this.

Is there some kind of intuitive argument for why the last sum should be zero?

  • 1
    $\begingroup$ It's the Pythagorean Theorem. Among many explanations on this site, this one has a picture. The thread at stats.stackexchange.com/questions/258284 seems to be a duplicate: does that answer your question? $\endgroup$
    – whuber
    Sep 25 '17 at 21:52
  • $\begingroup$ When you have an assumption of linearity ( Y = aX + b) , it will be true . The covariance will be zero. People appesr to term it as true score theory. To me it appears to have a basis in mathematical statistics. There is nothing like intution here. I am simply making half baked comments. I am a naive person - just responding. But the question is good. $\endgroup$ Sep 26 '17 at 14:25
  • $\begingroup$ @whuber: I totally missed those pages in my earlier search. (Not sure how.) Okay, so that makes a lot of sense to me. Of course, for my undergraduates in an introductory stats class, that page probably doesn't help a whole lot. (They have not had linear algebra and so we don't have the geometric picture of least squares available.) I guess that a question that asks for "intuition" depends pretty heavily on prerequisite knowledge used to guide said intuition, so it's probably an unfair question in some respects. Anyway, thanks for pointing me to the relevant resources. $\endgroup$ Sep 27 '17 at 5:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.