Skip to main content
1 of 3
Dave
  • 67.2k
  • 7
  • 105
  • 305

There are two common reasons why square loss (such as $MSE$) is popular.

  1. Large misses are brutally punished. If your residual is $1$, your squared residual is $1$, but if your residual is $2$, your squared residual is $4$, and if your residual is $10$, your squared residual is $100$. By increasing the residual by a factor of ten, the square loss penalty is increased by a factor of $100$.

  2. If we assume Gaussian error terms, the least squares solution is equivalent to maximum likelihood of the regression parameters.

(If others comment about other reasons why square loss is popular, I can edit to include them, but these are the ones that come to mind quickly.)

The way to compare which model does a better job of minimizing square loss is to look at which model minimizes such a value, so of course there is a sense in which $MSE$ is a legitimate performance metric.

A typical criticism of $R^2$ is that it can be driven arbitrarily high by overfitting, and this criticism is legitimate. However, measures of square loss like $MSE$ and $RMSE$ suffer from the same issue. If $R^2 = 1$ then $RMSE = 0$ and $MSE=0$.

A remedy for this is to do out-of-sample testing, such as the cross validation for which this Stack is named. While we might be able to drive $R^2$ up to $1$ by including features that are unrelated to the outcome and give a regression model that fits the noise instead of the signal (something like playing connect-the-dots with the scatterplot), out-of-sample performance will be poor when this is the case, hence the appeal of out-of-sample testing in machine learning.

There is an out-of-sample $R^2$:

$$ R^2_{out} = 1 - \dfrac{n_{out}MSE_{out}}{\sum_{i = 1}^{n_{out}}\big(y_i - \bar y_{in}\big)^2}\\ n_{out}\text{: Number of observations in the out-of-sample data}\\ MSE_{out}\text{: Mean of the squared residuals for the out-of-sample predictions}\\ \bar y_{in}\text{: Mean of the in-sample response variable} $$

Notice that the subscripts in that equation indicate out-of-sample numbers, except for the mean $\bar y_{in}$.

To understand why, consider what in-sample $R^2$ measures: a comparison of the model under consideration compared to a model that naïvely guesses the pooled mean of $y$ every time in its attempt to model the conditional mean. It makes sense to consider such a model to be the baseline when we test out-of-sample. If we cannot do better than just guessing $\bar y_{in}$ every time, we have done a poor job of modeling the conditional mean.

The denominator term of $R^2_{out}$ is constant for a given test set, regardless of the model. Consequently, maximizing $R^2_{out}$ is equivalent to minimizing $MSE$.

Dave
  • 67.2k
  • 7
  • 105
  • 305