Trying to understand this relationship I came across this conversation in an email group:

I got a problem while analyzing my data using NN. I got R-square with 0.45 with MSE around 5000. SO I am wonder why? There must be a relation between R-square and MSE?

R2 = 1 - SSE/SS0

SSE = N*MSE SS0 = (N-1)*VAR(Target)

VAR(T) = SS0/(N-1) = [SSE/(1-R2)]/(N-1)

= [N/N-1]*MSE/(1-R2)

~ 5000/0.55 ~ 9091

If you had standardized your targets to unit variance you would have obtained MSE ~ 0.55

Is that correct? What insights can we reach understanding this relationship?


Yes, allow me to elaborate.

Recall that for some outcome $y_i \in \mathbb{R}, \forall i=1,2,..,n$ we define MSE and $\textrm{R}^2$ as

\begin{equation} \textrm{MSE}(y, \hat{y} ) = \frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y_i})^{2} \end{equation}

\begin{equation} \textrm{R}^2(y, \hat{y} ) = 1 - \frac{\sum_{i=1}^{n} (y_i - \hat{y_i})^{2}}{ \sum_{i=1}^{n} (y_i - \bar{y})^{2} } \end{equation}

So, as you noted, $\textrm{R}^2$ is a normalized version of MSE, we use MSE for reporting because I think it's a simple metric and it is technically the loss-function we are minimizing when we solve the normal equations.

$\textrm{R}^2$ is useful because it is often easier to interpret since it doesn't depend on the scale of the data.

As a concrete example, consider two models: one predicting income and the other predicting age, $\textrm{R}^2$ will make it easier to state which model is performing better.*

*In general, this isn't a great idea and you shouldn't compare metrics like $\textrm{R}^2$ across different models to make these sorts of claims because some things are just fundamentally harder to predict than others (e.g., stock markets vs. who survived the Titanic).

  • $\begingroup$ Good call @C8H10N4O2! Update! $\endgroup$ – Francisco Arceo 2 days ago

Not the answer you're looking for? Browse other questions tagged or ask your own question.