I'm working on simple linear regression, and I would like to understand the relationship between RMSE and RSS (residual sum of squares).

In another Stackexchange question, I found some explanations, but they didn't directly explain the answer to my particular question, and definitely not in a way I could understand.

What is the relationship between RMSE and RSS in linear regression?

  • $\begingroup$ RMSE = RSS/sqrt(# of residuals), I presume. $\endgroup$
    – KH Kim
    Apr 9 '16 at 1:06

Having the mathematical derivations, you might ask yourself why use one measure over the other to assess the performance of a given model? You could use either, but the advantage of RMSE is that it will come out in more interpretable units. For example, if you were building a model that used house features to predict house prices, RSS would come out in dollars squared and would be a really huge number. RMSE would come out in dollars and its magnitude would make more sense given the range of your house price predictions.

  • The RSS is the sum of the square of the errors (difference between calculation and measurement, or estimated and real values):

$ RSS = \sum{(\hat Y_i-Y_i)^2} $

  • The MSE is the mean of that sum of the square of the errors:

$ MSE = \frac{1}{n}\sum{(\hat Y_i-Y_i)^2}$

  • The RMSE is the square root of the MSE:

$ RMSE = \sqrt{MSE} $

A bit of math shows:

$ RMSE = \sqrt{MSE} = \sqrt{\frac{1}{n} \cdot RSS} $

You can check it in the example that you posted:

$ RMSE = \sqrt{\frac{1}{32} \cdot 447.6743} = 3.740297 $

Note that for the mtcars dataset $n=32$.

Also see this question

  • $\begingroup$ Wait wait wait, then why is MSE not called MRS (Mean of Residual Squared)??? $\endgroup$
    – NoName
    Mar 24 '20 at 22:43
  • $\begingroup$ @NoName It was like that when I got here! ¯_(ツ)_/¯ $\endgroup$
    – Luis
    Mar 25 '20 at 15:59
  • $\begingroup$ While we're protesting naming conventions, why is RSS not called SSR (sum of square residuals)?! $\endgroup$
    – gwelter
    Oct 21 '20 at 22:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.