Confused with Residual Sum of Squares and Total Sum of Squares

Question

From Wikipedia: https://en.wikipedia.org/wiki/Residual_sum_of_squares, the RSS is the average squared error between true value $y$, and the predicted value $\hat y$.

Then according to: https://en.wikipedia.org/wiki/Total_sum_of_squares, the TSS is the squared error between the true value $y$, and the average of all $y$.

---

However, I don't understand this line under the explanation for TSS:

[...] the total sum of squares equals the explained sum of squares plus the residual sum of squares.

---

If we plot RSS on the graph, it would look like:

---

TSS Plot:

---

ESS Plot:

---

According to the images, the residual (unexplained) value is actually larger than the TSS. Is there something I'm not following?

Answer 1

You have the total sum of squares being $\displaystyle \sum_i ({y}_i-\bar{y})^2$

which you can write as $\displaystyle \sum_i ({y}_i-\hat{y}_i+\hat{y}_i-\bar{y})^2 $

i.e. as $\displaystyle \sum_i ({y}_i-\hat{y}_i)^2+2\sum_i ({y}_i-\hat{y}_i)(\hat{y}_i-\bar{y}) +\sum_i(\hat{y}_i-\bar{y})^2$ where

• the first summation term is the residual sum of squares,
• the second is zero (if not then there is correlation, suggesting there are better values of $\hat{y}_i$) and
• the third is the explained sum of squares

Since you have sums of squares, they must be non-negative and so the residual sum of squares must be less than the total sum of squares

Answer 2

This chart was very helpful for me.

Via