1
$\begingroup$

I understand that in order to have a good, stable model the $R^2$ has to be high and the RMSE must be low (depending on the type of variables examined). There are many questions about the "best RMSE", but my question is about something different.

Are there any possible causes for high $R^2$ values AND high RMSE values? How can I find more about this and thus explain the results?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

The two are directly related:

$R^2 = 1-\frac{\sum(y_i-\hat{y})^2}{\sum(y_i-\bar{y})^2}$

$RMSE = \sqrt{\frac{\sum(y_i-\hat{y})}{n-k}^2}$

so

$R^2 = 1-\frac{RMSE^2\times(n-k)}{\sum(y_i-\bar{y})^2}$

Now the unit of the RMSE is the unit of the dependent variable, while the $R^2$ is a proportion. So, numerically you can arbitrarily change RMSE while keeping the $R^2$ constant by changing the unit of the dependent variable. Such a change has no substantive meaning; you can say that something weights a 1000 grams or 1 kg, the numbers are different but the meaning is exactly the same.

Improve this answer
$\endgroup$
10
  • $\begingroup$ I think you forgot to take mean instead of sum. $\endgroup$
    – Richard Hardy
    Commented Feb 23, 2017 at 9:58
  • $\begingroup$ @Maarten Buis thank you for your response. Υou're right about this but since there is a clear relation , I can not understand what causes a result like R2=0.80 and RMSE=200 (both high). Would it be reasonable something like R2= 0.80 and RMSE= 0.5? I just can not see any possible reasons. $\endgroup$
    – g.f.l
    Commented Feb 23, 2017 at 10:02
  • $\begingroup$ @g.f.l You cannot say if 200 for a RMSE is high or low, that crucially depends on the unit of the dependent variable. $\endgroup$
    – Maarten Buis
    Commented Feb 23, 2017 at 10:06
  • $\begingroup$ @MaartenBuis exactly, I said that in my question (depending on the type of variables examined). But if it is high, is there something I can read to explain this and find possible causes? I only find something like stats.stackexchange.com/questions/56302/… $\endgroup$
    – g.f.l
    Commented Feb 23, 2017 at 10:11
  • $\begingroup$ Maybe an example helps? Lets say your dependent variable is weight, and you measure it once in grams and once in kg. The RMSE in your first model will be 1000 times the RMSE in your second model. So an RMSE of 200 in the first model will correspond to an RMSE of .2 in the second. Now I don't think there is a problem here that needs explaining other than that 1 kg=1000 gr. $\endgroup$
    – Maarten Buis
    Commented Feb 23, 2017 at 10:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.