What is the mathematical relationship between R2 and MSE? Trying to understand this relationship I came across this conversation in an email group: 


I got a problem while analyzing my data using NN.
     I got R-square with 0.45 with MSE around 5000.
    SO I am wonder why? There must be a relation
    between R-square and MSE?

R2 = 1 - SSE/SS0
SSE = N*MSE SS0 = (N-1)*VAR(Target)
VAR(T) = SS0/(N-1) = [SSE/(1-R2)]/(N-1)
= [N/N-1]*MSE/(1-R2)
~ 5000/0.55 ~ 9091
If you had standardized your targets to unit variance you would have
  obtained MSE ~ 0.55

Is that correct?
What insights can we reach understanding this relationship? 
 A: Yes, allow me to elaborate.
Recall that for some outcome $y_i \in \mathbb{R}, \forall i=1,2,..,n$ we define MSE and $\textrm{R}^2$ as
\begin{equation}
\textrm{MSE}(y, \hat{y} ) = \frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y_i})^{2}
\end{equation}
\begin{equation}
\textrm{R}^2(y, \hat{y} ) = 1 - \frac{\sum_{i=1}^{n} (y_i - \hat{y_i})^{2}}{ \sum_{i=1}^{n} (y_i - \bar{y})^{2} }
\end{equation}
So, as you noted, $\textrm{R}^2$ is a normalized version of MSE, we use MSE for reporting because I think it's a simple metric and it is technically the loss-function we are minimizing when we solve the normal equations.
$\textrm{R}^2$ is useful because it is often easier to interpret since it doesn't depend on the scale of the data.
As a concrete example, consider two models: one predicting income and the other predicting age, $\textrm{R}^2$ will make it easier to state which model is performing better.*

*In general, this isn't a great idea and you shouldn't compare metrics like $\textrm{R}^2$ across different models to make these sorts of claims because some things are just fundamentally harder to predict than others (e.g., stock markets vs. who survived the Titanic).
