Skip to main content
Cross Validated

Return to Answer

added 10 characters in body
Source Link
Dave
Dave
  • 67.2k
  • 7
  • 105
  • 305

$R^2$ forces you to compare model performance to the performance of a baseline model.

$$ R^2=1-\left(\dfrac{ \overset{N}{\underset{i=1}{\sum}}\left( y_i-\hat y_i \right)^2 }{ \overset{N}{\underset{i=1}{\sum}}\left( y_i-\bar y \right)^2 }\right) $$

The numerator is a function of the $\text{RMSE}$ of your model (square it and the multiply by the sample size, $N$). The denominator is that same function of the $\text{RMSE}$ of a model that always predicts $\bar y$, which is a reasonable baseline to which performance can be compared: if you want to predict the conditional mean and have no idea how to do that, what better option than predicting the marginal/pooled mean $\bar y$ every time?

$$ R^2=1-\left(\dfrac{ \overset{N}{\underset{i=1}{\sum}}\left( y_i-\hat y_i \right)^2 }{ \overset{N}{\underset{i=1}{\sum}}\left( y_i-\bar y \right)^2 }\right) =1 - \left(\dfrac{ N\times \left(\text{RMSE}_{\text{model}}\right)^2 }{ N\times \left(\text{RMSE}_{\bar y}\right)^2 }\right) =1 - \left(\dfrac{ \left(\text{RMSE}_{\text{model}}\right)^2 }{ \left(\text{RMSE}_{\bar y}\right)^2 }\right) $$

Since $R^2$ forces you to compare to a benchmark, you avoid making silly claims just because the $\text{RMSE}$ appears to be a small number. Sure, the number might be small, but if you would have an even bettersmaller (better) $\text{RMSE}$ withusing a simplebasic model, you probably want to know if all of the hard work you've put in to develop your model has resulted in worse performance than if you just predicted the same $\bar y$ every time.

Yes, you will get similar information by just looking at the model $\text{RMSE}$ and comparing it to the $\text{RMSE}$ of a model that always predicts $\bar y$, but calculating $R^2$ explicitly forces you to do this.

A drawback of $R^2$ is that it is easy to get into a trap of looking at values like letter grades in school, where $R^2 = 0.95$ is an $\text{A}$ that makes you happy and $R^2 = 0.50$ is an $\text{F}$ that makes you sad. If the state-of-the-art in modeling only achieves $R^2 = 0.30$, then your $R^2 = 0.50$ doesn't sound so bad, and if people are routinely scoring $R^2 > 0.99$, then your $R^2 = 0.95$ doesn't sound so great. Out of context, it is difficult to consider model performance as good or bad, yet $R^2$ can give the illusion of aligning with letter grades.

$R^2$ forces you to compare model performance to the performance of a baseline model.

$$ R^2=1-\left(\dfrac{ \overset{N}{\underset{i=1}{\sum}}\left( y_i-\hat y_i \right)^2 }{ \overset{N}{\underset{i=1}{\sum}}\left( y_i-\bar y \right)^2 }\right) $$

The numerator is a function of the $\text{RMSE}$ of your model (square it and the multiply by the sample size, $N$). The denominator is that same function of the $\text{RMSE}$ of a model that always predicts $\bar y$, which is a reasonable baseline to which performance can be compared: if you want to predict the conditional mean and have no idea how to do that, what better option than predicting the marginal/pooled mean $\bar y$ every time?

$$ R^2=1-\left(\dfrac{ \overset{N}{\underset{i=1}{\sum}}\left( y_i-\hat y_i \right)^2 }{ \overset{N}{\underset{i=1}{\sum}}\left( y_i-\bar y \right)^2 }\right) =1 - \left(\dfrac{ N\times \left(\text{RMSE}_{\text{model}}\right)^2 }{ N\times \left(\text{RMSE}_{\bar y}\right)^2 }\right) =1 - \left(\dfrac{ \left(\text{RMSE}_{\text{model}}\right)^2 }{ \left(\text{RMSE}_{\bar y}\right)^2 }\right) $$

Since $R^2$ forces you to compare to a benchmark, you avoid making silly claims just because the $\text{RMSE}$ appears to be a small number. Sure, the number might be small, but if you would have an even better $\text{RMSE}$ with a simple model, you probably want to know if all of the hard work you've put in to develop your model has resulted in worse performance than if you just predicted the same $\bar y$ every time.

Yes, you will get similar information by just looking at the model $\text{RMSE}$ and comparing it to the $\text{RMSE}$ of a model that always predicts $\bar y$, but calculating $R^2$ explicitly forces you to do this.

A drawback of $R^2$ is that it is easy to get into a trap of looking at values like letter grades in school, where $R^2 = 0.95$ is an $\text{A}$ that makes you happy and $R^2 = 0.50$ is an $\text{F}$ that makes you sad. If the state-of-the-art in modeling only achieves $R^2 = 0.30$, then your $R^2 = 0.50$ doesn't sound so bad, and if people are routinely scoring $R^2 > 0.99$, then your $R^2 = 0.95$ doesn't sound so great. Out of context, it is difficult to consider model performance as good or bad, yet $R^2$ can give the illusion of aligning with letter grades.

$R^2$ forces you to compare model performance to the performance of a baseline model.

$$ R^2=1-\left(\dfrac{ \overset{N}{\underset{i=1}{\sum}}\left( y_i-\hat y_i \right)^2 }{ \overset{N}{\underset{i=1}{\sum}}\left( y_i-\bar y \right)^2 }\right) $$

The numerator is a function of the $\text{RMSE}$ of your model (square it and the multiply by the sample size, $N$). The denominator is that same function of the $\text{RMSE}$ of a model that always predicts $\bar y$, which is a reasonable baseline to which performance can be compared: if you want to predict the conditional mean and have no idea how to do that, what better option than predicting the marginal/pooled mean $\bar y$ every time?

$$ R^2=1-\left(\dfrac{ \overset{N}{\underset{i=1}{\sum}}\left( y_i-\hat y_i \right)^2 }{ \overset{N}{\underset{i=1}{\sum}}\left( y_i-\bar y \right)^2 }\right) =1 - \left(\dfrac{ N\times \left(\text{RMSE}_{\text{model}}\right)^2 }{ N\times \left(\text{RMSE}_{\bar y}\right)^2 }\right) =1 - \left(\dfrac{ \left(\text{RMSE}_{\text{model}}\right)^2 }{ \left(\text{RMSE}_{\bar y}\right)^2 }\right) $$

Since $R^2$ forces you to compare to a benchmark, you avoid making silly claims just because the $\text{RMSE}$ appears to be a small number. Sure, the number might be small, but if you would have an even smaller (better) $\text{RMSE}$ using a basic model, you probably want to know if all of the hard work you've put in to develop your model has resulted in worse performance than if you just predicted the same $\bar y$ every time.

Yes, you will get similar information by just looking at the model $\text{RMSE}$ and comparing it to the $\text{RMSE}$ of a model that always predicts $\bar y$, but calculating $R^2$ explicitly forces you to do this.

A drawback of $R^2$ is that it is easy to get into a trap of looking at values like letter grades in school, where $R^2 = 0.95$ is an $\text{A}$ that makes you happy and $R^2 = 0.50$ is an $\text{F}$ that makes you sad. If the state-of-the-art in modeling only achieves $R^2 = 0.30$, then your $R^2 = 0.50$ doesn't sound so bad, and if people are routinely scoring $R^2 > 0.99$, then your $R^2 = 0.95$ doesn't sound so great. Out of context, it is difficult to consider model performance as good or bad, yet $R^2$ can give the illusion of aligning with letter grades.

Source Link
Dave
Dave
  • 67.2k
  • 7
  • 105
  • 305

$R^2$ forces you to compare model performance to the performance of a baseline model.

$$ R^2=1-\left(\dfrac{ \overset{N}{\underset{i=1}{\sum}}\left( y_i-\hat y_i \right)^2 }{ \overset{N}{\underset{i=1}{\sum}}\left( y_i-\bar y \right)^2 }\right) $$

The numerator is a function of the $\text{RMSE}$ of your model (square it and the multiply by the sample size, $N$). The denominator is that same function of the $\text{RMSE}$ of a model that always predicts $\bar y$, which is a reasonable baseline to which performance can be compared: if you want to predict the conditional mean and have no idea how to do that, what better option than predicting the marginal/pooled mean $\bar y$ every time?

$$ R^2=1-\left(\dfrac{ \overset{N}{\underset{i=1}{\sum}}\left( y_i-\hat y_i \right)^2 }{ \overset{N}{\underset{i=1}{\sum}}\left( y_i-\bar y \right)^2 }\right) =1 - \left(\dfrac{ N\times \left(\text{RMSE}_{\text{model}}\right)^2 }{ N\times \left(\text{RMSE}_{\bar y}\right)^2 }\right) =1 - \left(\dfrac{ \left(\text{RMSE}_{\text{model}}\right)^2 }{ \left(\text{RMSE}_{\bar y}\right)^2 }\right) $$

Since $R^2$ forces you to compare to a benchmark, you avoid making silly claims just because the $\text{RMSE}$ appears to be a small number. Sure, the number might be small, but if you would have an even better $\text{RMSE}$ with a simple model, you probably want to know if all of the hard work you've put in to develop your model has resulted in worse performance than if you just predicted the same $\bar y$ every time.

Yes, you will get similar information by just looking at the model $\text{RMSE}$ and comparing it to the $\text{RMSE}$ of a model that always predicts $\bar y$, but calculating $R^2$ explicitly forces you to do this.

A drawback of $R^2$ is that it is easy to get into a trap of looking at values like letter grades in school, where $R^2 = 0.95$ is an $\text{A}$ that makes you happy and $R^2 = 0.50$ is an $\text{F}$ that makes you sad. If the state-of-the-art in modeling only achieves $R^2 = 0.30$, then your $R^2 = 0.50$ doesn't sound so bad, and if people are routinely scoring $R^2 > 0.99$, then your $R^2 = 0.95$ doesn't sound so great. Out of context, it is difficult to consider model performance as good or bad, yet $R^2$ can give the illusion of aligning with letter grades.