Suppose I'm doing regression with training, validation, and test sets. I can find RMSE and R squared (R^2, the coefficient of determination) from the output of my software (such as R's lm() function).

My understanding is that the test RMSE (or MSE) is the measure of goodness of predicting the validation/test values, while R^2 is a measure of goodness of fit in capturing the variance in the training set.

In the real world, what I really care about is generalized prediction accuracy on data I haven't seen. So then what is the utility of the R^2 value compared to RMSE?


The unadjusted $R^2$ is defined to be $$R^2 = 1 - \frac{\frac{1}{n}\sum_{i=1}^n (y_i - \hat y_i)^2}{\frac{1}{n}\sum_{i=1}^n (y_i - \bar y)^2} = 1 - \frac{MSE}{\frac{1}{n}TotSS}$$

Let's take the RMSE to be $$ RMSE = \sqrt{MSE}. $$

For a given data set $y_i$ and $\bar y$ are fixed, so as different models are considered only the $\hat y_i$ change. This means that in the above expressions, only the MSE changes. So both $R^2$ and $RMSE$ are functions of the same thing, and therefore there isn't much of a difference (except for interpretation) by considering one versus the other.

If we instead look at the adjusted $R^2$ or use $RMSE = \sqrt{\frac{n}{n-p}MSE}$ then we'll also have $p$, the dimension of the model, changing for different models.

| cite | improve this answer | |

Chaconne did an excellent job about defining the measures formulas and how they are very closely related from a math standpoint. If you benchmark or rank models using the same data set those two measures are interchangeable, meaning you will get the exact same ranking of your models whether you use R Square (ranking them high to low) or the RMSE (ranking them low to high).

However, the two measures have a very different meaning and use. R Square is not only a measure of Goodness-of-fit, it is also a measure of how much the model (the set of independent variables you selected) explain the behavior (or the variance) of your dependent variable. So, if your model has an R Square of 0.60, it explains 60% of the behavior of your dependent variable. Now, if you use the Adjusted R Square that essentially penalizes the R Square for the number of variables you use you get a pretty good idea when you should stop adding variables to your model (and eventually just get a model that is overfit). If your Adjusted R Square is 0.60. And, when you add an extra variable it just increases to 0.61. It is probably not worth it adding this extra variable.

Now, turning to RMSE also most commonly referred to as a Standard Error. It has a completely different use than R Square. The Standard Error allows you to build Confidence Intervals around your regression estimate assuming whatever Confidence Level you are interested in (typically 99%, 95%, or 90%). Indeed, the Standard Error is the equivalent of a Z value. So, if you want to build a 95% CI around your regression trendline you multiply the Standard Error by 1.96 and quickly generates a high and low estimate as border of your 95% CI around the regression line.

So, both R Square (and Adjusted R Square) and the Standard Error are extremely useful in assessing the statistical robustness of a model. And, as indicated they have completely different practical application. One measures the explanatory power of the model. The other one allows you to build Confidence Intervals. Both, very useful but different stuff.

Regarding assessing prediction accuracy on data you have not seen, both measures have their limitations as well as most other measures you can think off. On new data that is out-of-sample, the R Square and Standard Error on the history or learning sample of the model will not be of much use. The out-of-sample stuff is just a great test to check whether your model is overfit (great R Square and low Standard Error, but poor performance in out-of-sample) or not. I understand better measures for prospective data (data you have not seen yet) are the information criterion including AIC, BIC, SIC. And, the model with the best information criterion values should handle unseen data better, in other words be more predictive. Those measures are close cousins of the Adjusted R Square concept. However, they are more punitive on adding additional variables than Adjusted R Square is.

| cite | improve this answer | |
  • 1
    $\begingroup$ Thanks for your answer. I have typically used RMSE only for assessing the predictive power of a linear regression model (after predicting the values of an unseen test set). So I've not seen that RMSE "has a completely different use ... to build confidence intervals around your regression estimate." I guess this must be a statistician thing? I'm from computer science, so I haven't computed very many confidence intervals in my career. $\endgroup$ – stackoverflowuser2010 May 28 '16 at 18:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.