From Wikipedia: https://en.wikipedia.org/wiki/Residual_sum_of_squares, the RSS is the average squared error between true value $y$, and the predicted value $\hat y$.

Then according to: https://en.wikipedia.org/wiki/Total_sum_of_squares, the TSS is the squared error between the true value $y$, and the average of all $y$.

However, I don't understand this line under the explanation for TSS:

[...] the total sum of squares equals the explained sum of squares plus the residual sum of squares.

If we plot RSS on the graph, it would look like:

enter image description here

TSS Plot:

enter image description here

ESS Plot:

enter image description here

According to the images, the residual (unexplained) value is actually larger than the TSS. Is there something I'm not following?


2 Answers 2


You have the total sum of squares being $\displaystyle \sum_i ({y}_i-\bar{y})^2$

which you can write as $\displaystyle \sum_i ({y}_i-\hat{y}_i+\hat{y}_i-\bar{y})^2 $

i.e. as $\displaystyle \sum_i ({y}_i-\hat{y}_i)^2+2\sum_i ({y}_i-\hat{y}_i)(\hat{y}_i-\bar{y}) +\sum_i(\hat{y}_i-\bar{y})^2$ where

  • the first summation term is the residual sum of squares,
  • the second is zero (if not then there is correlation, suggesting there are better values of $\hat{y}_i$) and
  • the third is the explained sum of squares

Since you have sums of squares, they must be non-negative and so the residual sum of squares must be less than the total sum of squares

  • 1
    $\begingroup$ Hi, thanks for your answer. It may be a bit too late, but can you elaborate on how the 2nd term means mathematically? Looking at the equation, I don't think I can come up with the conclusion that it needs to be 0. Thanks again! $\endgroup$ Commented Mar 9, 2017 at 7:50
  • $\begingroup$ Can I say this for the second term: For linear regression, the amount we over estimate and under estimate is zero, so when we sum $y_i$-$\hat y_i$, we will get 0? $\endgroup$ Commented Mar 9, 2017 at 18:29
  • $\begingroup$ Yes you should get $0$ since the average of each should be $\bar{y}$ and so they have the same sums. But my point is subtly different: the residuals should be uncorrelated with the predicted values as a direct result of the linear regression $\endgroup$
    – Henry
    Commented Mar 9, 2017 at 18:35
  • 2
    $\begingroup$ $\sum_i ({y}_i-\hat{y}_i)(\hat{y}_i-\bar{y})$ is the covariance of the residuals and the predicted values. Try an ordinary least squares regression and see that it is zero $\endgroup$
    – Henry
    Commented Mar 9, 2017 at 21:15
  • 1
    $\begingroup$ Do you mean $cov((y_i- \hat y_i), \hat y_i)$? Thanks again, I'm trying to work out the math details to fully understand what is going on. $\endgroup$ Commented Mar 9, 2017 at 21:43

This chart was very helpful for me. enter image description here


  • 1
    $\begingroup$ could you please reference chart? $\endgroup$ Commented Aug 3, 2018 at 11:39
  • 1
    $\begingroup$ Excellent explanation! Thank you! $\endgroup$
    – Beginner
    Commented Nov 19, 2021 at 16:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.