6
$\begingroup$

I'm working on simple linear regression, and I would like to understand the relationship between RMSE and RSS (residual sum of squares).

In another Stackexchange question, I found some explanations, but they didn't directly explain the answer to my particular question, and definitely not in a way I could understand.

What is the relationship between RMSE and RSS in linear regression?

$\endgroup$
1
  • $\begingroup$ RMSE = RSS/sqrt(# of residuals), I presume. $\endgroup$
    – KH Kim
    Apr 9, 2016 at 1:06

2 Answers 2

4
$\begingroup$

Having the mathematical derivations, you might ask yourself why use one measure over the other to assess the performance of a given model? You could use either, but the advantage of RMSE is that it will come out in more interpretable units. For example, if you were building a model that used house features to predict house prices, RSS would come out in dollars squared and would be a really huge number. RMSE would come out in dollars and its magnitude would make more sense given the range of your house price predictions.

$\endgroup$
17
$\begingroup$
  • The RSS is the sum of the square of the errors (difference between calculation and measurement, or estimated and real values):

$ RSS = \sum{(\hat Y_i-Y_i)^2} $

  • The MSE is the mean of that sum of the square of the errors:

$ MSE = \frac{1}{n}\sum{(\hat Y_i-Y_i)^2}$

  • The RMSE is the square root of the MSE:

$ RMSE = \sqrt{MSE} $

A bit of math shows:

$ RMSE = \sqrt{MSE} = \sqrt{\frac{1}{n} \cdot RSS} $

You can check it in the example that you posted:

$ RMSE = \sqrt{\frac{1}{32} \cdot 447.6743} = 3.740297 $

Note that for the mtcars dataset $n=32$.


Also see this question

$\endgroup$
4
  • $\begingroup$ Wait wait wait, then why is MSE not called MRS (Mean of Residual Squared)??? $\endgroup$
    – NoName
    Mar 24, 2020 at 22:43
  • 2
    $\begingroup$ @NoName It was like that when I got here! ¯_(ツ)_/¯ $\endgroup$
    – Luis
    Mar 25, 2020 at 15:59
  • $\begingroup$ While we're protesting naming conventions, why is RSS not called SSR (sum of square residuals)?! $\endgroup$
    – gwelter
    Oct 21, 2020 at 22:46
  • $\begingroup$ @gwelter Because sum of square residual and residual sum of square mean the same thing. But mean of squared error and mean of squared residual can mean very different thing as in regression residual is used to estimate error. But to get back to the point I have the same question as @NoName. $\endgroup$ Jul 30, 2022 at 22:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.