# Confused with Residual Sum of Squares and Total Sum of Squares

From Wikipedia: https://en.wikipedia.org/wiki/Residual_sum_of_squares, the RSS is the average squared error between true value $y$, and the predicted value $\hat y$.

Then according to: https://en.wikipedia.org/wiki/Total_sum_of_squares, the TSS is the squared error between the true value $y$, and the average of all $y$.

However, I don't understand this line under the explanation for TSS:

[...] the total sum of squares equals the explained sum of squares plus the residual sum of squares.

If we plot RSS on the graph, it would look like:

TSS Plot:

ESS Plot:

According to the images, the residual (unexplained) value is actually larger than the TSS. Is there something I'm not following?

You have the total sum of squares being $\displaystyle \sum_i ({y}_i-\bar{y})^2$

which you can write as $\displaystyle \sum_i ({y}_i-\hat{y}_i+\hat{y}_i-\bar{y})^2$

i.e. as $\displaystyle \sum_i ({y}_i-\hat{y}_i)^2+2\sum_i ({y}_i-\hat{y}_i)(\hat{y}_i-\bar{y}) +\sum_i(\hat{y}_i-\bar{y})^2$ where

• the first summation term is the residual sum of squares,
• the second is zero (if not then there is correlation, suggesting there are better values of $\hat{y}_i$) and
• the third is the explained sum of squares

Since you have sums of squares, they must be non-negative and so the residual sum of squares must be less than the total sum of squares

• Hi, thanks for your answer. It may be a bit too late, but can you elaborate on how the 2nd term means mathematically? Looking at the equation, I don't think I can come up with the conclusion that it needs to be 0. Thanks again! – user1157751 Mar 9 '17 at 7:50
• Can I say this for the second term: For linear regression, the amount we over estimate and under estimate is zero, so when we sum $y_i$-$\hat y_i$, we will get 0? – user1157751 Mar 9 '17 at 18:29
• Yes you should get $0$ since the average of each should be $\bar{y}$ and so they have the same sums. But my point is subtly different: the residuals should be uncorrelated with the predicted values as a direct result of the linear regression – Henry Mar 9 '17 at 18:35
• Thanks for your reply again! I don't see a connection between "the residuals should be uncorrelated with the predicted values as a direct result of the linear regression" with any of the terms (1, 2, and 3). Can you give me some hints? Thanks – user1157751 Mar 9 '17 at 18:44
• $\sum_i ({y}_i-\hat{y}_i)(\hat{y}_i-\bar{y})$ is the covariance of the residuals and the predicted values. Try an ordinary least squares regression and see that it is zero – Henry Mar 9 '17 at 21:15

This chart was very helpful for me.

• could you please reference chart? – Carlos S Traynor Aug 3 '18 at 11:39