Explained variance score vs $R^2$ score

Question

I came across explained variance score and $R^2$ score in scikit learn docs.

Docs defines exaplained variance score as:

$\text{explained variance} (y,\hat{y})=1-\frac{Var\{y-\hat{y}\}}{Var\{y\}}$

which I feel equates to $$1-\frac{\sum(y-\hat{y}-\overline{(y-\hat{y})})^2}{\sum(y-\bar{y})^2}\tag I$$

Docs then define $R^2$ score as follows:

$$R^2(y,\hat{y})=1-\frac{\sum(y-\hat{y})^2}{\sum(y-\bar{y})^2} \tag{II}$$

The difference between equation $\rm (I)$ and $\rm (II)$ is that equation $\rm (I)$ has $"-\overline{(y-\hat{y})}"$ in the numerator, while equation $\rm (II)$ does not. I have learnt from the various sources what different values of $R^2$ indicate:

• $R^2<0→$ horizontal line explains the data better than the model, in other words, the chosen model does not follow the trend of the data
• $R^2=0→$ horizontal line explains the data as well as the model, i.e. model has no explanatory value
• $R^2>0→$ model explains the data better than the horizontal line
• $R^2=1→$ means the model / regression line explains 100% of the variation in the response variable. This will be the case when all the data points lie precisely on the regression line

Q1. What to make sense of different values of "explained variance" and why?
Q2. Also, the doc points to this wiki page which does not give formula in equation $\rm (I)$. Why so?
Q3. How the two quantities are related intuitively?

Answer 1

The explained variance formula compares the variance of your residuals to the variance, which sounds great. However, the documentation says exactly why such a metric is not as helpful as it might appear.

The difference between the explained variance score and the R² score, the coefficient of determination is that when the explained variance score does not account for systematic offset in the prediction.

For instance, the explained variance function will see something like $y=(1,2,3,4)$ and $\hat y = (8, 9, 10, 11)$ and give a perfect score despite the predictions being awful. The formula $(II)$ would correctly flag this as problematic.

formula_1 <- function(y, yhat){
  ybar <- mean(y)
  return(
    1
    -
      var(y - yhat)
    /
      var(y - ybar)
  )
}
formula_2 <- function(y, yhat){
  ybar <- mean(y)
 return(
   1
   -
     sum((y - yhat)^2)
   /
     sum((y - ybar)^2)
 ) 
}
y <- c(1, 2, 3, 4)
yhat <- y + 7
formula_1(y, yhat) # I get 1, indicating a perfect fit
formula_2(y, yhat) # I get -38.2, indicating a terrible fit

This comes down to the fact that for any random variable $X$ with a finite variance and any $r\in\mathbb R$, $\text{var}(X) = \text{var}(X + r)$, yet $X = X+r$ does not hold in general, and this explained variance measure of performance does not catch such situations (which the sklearn $R^2$ function will catch).

Where the explained variance score could be useful is if you compare it to the $R^2$. If explained variance is good but $R^2$ is not, it could indicate systematic bias in the predictions, and you might be able to remedy this.

Note that the two can coincide, such as for in-sample predictions for an OLS linear regression.

set.seed(2023)
N <- 1000
formula_1 <- function(y, yhat){
  ybar <- mean(y)
  return(
    1
    -
      var(y - yhat)
    /
      var(y - ybar)
  )
}
formula_2 <- function(y, yhat){
  ybar <- mean(y)
  return(
    1
    -
      sum((y - yhat)^2)
    /
      sum((y - ybar)^2)
  ) 
}
x <- runif(N, -2, 2)
y <- x + rnorm(N) + 7
L <- lm(y ~ x)
yhat <- predict(L)
formula_1(y, yhat) # I get 0.5379571
formula_2(y, yhat) # I get 0.5379571