Why can $R^2$ be negative in linear regression -- interview question I was asked an $R^2$ question during an interview, and I felt like I was right then, and still feel like I'm right now. Essentially the interviewer asked me if it is possible for $R^2$ to be negative for linear regression.
I said that if you're using OLS, then it is not possible because the formal definition of $R^2$ is
$$
R^2 = 1 - \frac{SS_{res}}{SS_{tot}}
$$
where $SS_{tot} = \sum_i^n (y_i - \bar{y})^2$ and  $SS_{res} = \sum_i^n (y_i - \hat{y_i})^2$.
In order for $R^2$ to be negative, the second term must be greater than 1. This would imply that $SS_{res} > SS_{tot}$, which would imply that the predictive model fits worse than if you fit a straight line through the mean of the observed $y$.
I told the interviewer that it is not possible for $R^2$ to be 1 because if the horizontal line is indeed the line of best fit, then OLS fill produce that line unless we're dealing with an ill-conditioned or singular system.
He claimed that this isn't correct and that $R^2$ can still be negative, and that I could "see it easily in the case where there is no intercept." (note that all of the discussion so far was about the case WITH an intercept, which I confirmed at the beginning by asking if there are any constraints about the best line passing through the origin, which he stated "no")
I can't see this at all. I stood by my answer, and then mentioned that maybe if you used some other linear regression method, perhaps you can get a negative $R^2$.
Is there any way for $R^2$ to be negative using OLS with or without intercept? Edit: I do understand that you can get a negative $R^2$ in the case without an intercept.
 A: The interviewer is right. Sorry.
set.seed(2020)
x <- seq(0, 1, 0.001)
err <- rnorm(length(x))
y <- 99 - 30*x + err
L <- lm(y~0+x) # "0" forces the intercept to be zero
plot(x, y, ylim=c(0, max(y)))
abline(a=0, b= summary(L)$coef[1], col='red')
abline(h=mean(y), col='black')
SSRes <- sum(resid(L)^2)
SSTot <- sum((y - mean(y))^2)
R2 <- 1 - SSRes/SSTot
R2 

I get $R^2 = -31.22529$. This makes sense when you look at the plot the code produces.

The red line is the regression line. The black line is the "naive" line where you always guess the mean of $y$, regardless of the $x$.
The $R^2<0$ makes sense when you consider what $R^2$ does. $R^2$ measures how much better the regression model is at guessing the conditional mean than always guessing the pooled mean. Looking at the graph you're better off guessing the mean of the pooled values of $y$ than you are using the regression line.
EDIT
There is an argument to be made that the "SSTot" to which you should compare an intercept-free model is just the sum of squares of $y$ (so $\sum (y_i-0)^2$), not $\sum (y_i - \bar{y})^2$. However, $R^2_{ish} = 1- \frac{\sum(y_i - \hat{y}_i)^2}{\sum y_i^2}$ is quite different from the usual $R^2$ and (I think) loses the usual connection to amount of variance explained. If this $R^2_{ish}$ is used, however, when the intercept is excluded, $R^2_{ish} \ge 0$.
A: It looks like your interview was correct.
In the case that you include an intercept it is not possible.
The easiest way to see this is to take the projection view of linear regression.
$\hat{y} = X\hat{\beta} = X(X^TX)^{-1}X^TY = P_XY$
Where $P_X$ is a orthogonal projection matrix. It projects vectors into the subspace spanned by linear combinations of $X$. You can think of this as shining a light on the vector into the linear subspace spanned by X. It maps $Y$ to the closest possible part of the subspace.
We can also define the projection onto a subspace spanned by an intercept, denoted $P_\iota$, where $\iota$ is a vector of ones.
It turns out that $P_\iota Y = \bar{y}$, a $n \times 1$ vector with the mean as each value. In other words, the best possible linear approximation to $Y$ using only combinations of constants would be the mean. That makes sense and you may have seen related results in a stats class before.
If $X$ includes an intercept then the linear subspace spanned by $X$ is a superset of the linear subspace spanned by an intercept. What this means is that since $P_X$ finds the closest approximation in the subspace and it contains the intercept subspace, then it has to be at least as close to $Y$ as the best approximation in the span of $\iota$. In other words $|Y - \hat{y}|  = |Y - P_XY|  \leq |Y - P_\iota Y| = |Y - \bar{y}|$ if $X$ contains the intercept (and thus the squares must also follow this inequality).
Now if we do not include an intercept, this is no longer true, because the linear span of $X$ is no longer a superset of the intercept linear space. It is thus no longer guaranteed that our prediction is at least as good as the mean.
Consider the example where $X$ is a single variable with mean 0, finite variance and is independent of $Y$, and $Y$ has some arbitrary mean $E[Y] \neq 0$ (but exists).
$\hat{\beta} = (X^TX)^{-1}X^TY \overset{p}{\to} \frac{ E[XY] }{ E[X^2] } = \frac{E[X]E[Y]}{E[X^2]} = 0$
As n gets large, the coefficient becomes arbitrarily close to zero. This means that $\hat{y} \overset{p}{\to} 0$
Using the centered $\mathcal{R}^2$ formula we get
\begin{align}
1 - \frac{\sum_{i=1}^n (y_i - \hat{y})^2}{\sum_{i=1}^n(y_i -\bar{y})^2} &= 1 - \frac{\sum_{i=1}^n (y_i - o_p(1))^2}{\sum_{i=1}^n(y_i -\bar{y})^2}\\
&\overset{p}{\to} 1 - \frac{E[Y^2]}{var(Y)}\\
& = 1 - \frac{E[Y^2]}{E[Y^2] - (E[Y])^2} \leq 0
\end{align}
So if $X$ doesn't really explain anything in $Y$, and the mean of $Y$ is far from 0, we can have a really negative $\mathcal{R}^2$
Below is some R code to simulate such a case
set.seed(2020)
n <- 10000  
y <- rnorm(n,50,1)  
x <- rnorm(n)  

mod <- lm(y ~   -1 + x)  
yhat <- predict(mod)  

R2  <- 1 - sum((y - yhat)^2)/sum((y - mean(y))^2)  
R2

$\mathcal{R^2} = -2514.479$
Edit: I agree with Dave that when we don't include an intercept it would be reasonable to argue that the uncentered $\mathcal{R}^2$ is the more natural $\mathcal{R}^2$ measure. The problem with the uncentered version is that it is not invariant to changes in the mean of the regressand (see Davidson and Mackinnon: Econometric Theory and Methods chapter 3 for discussion).
A: Using OLS with intercept, the only situation with negative R-squared is the following:

*

*You fit your model on a training set.


*You apply the model on a fresh test set, calculate the out-of-sample residuals and from there, derive the out-of-sample R-squared. The latter can be negative.
Here the dummy example in R
n <- 100
df <- data.frame(x=rnorm(n), y=rnorm(n))
train <- df[1:70, ]
test <- df[71:n, ]

# Train on train
fit <- lm(y~x, train)
summary(fit) # Multiple R-squared:  3.832e-06

# Evaluate on test
oos_residuals <- test[, "y"] - predict(fit, test)

oos_residual_ss <- sum(oos_residuals^2)
oos_total_ss <- sum((test[, "y"] - mean(train[, "y"]))^2)

1 - oos_residual_ss / oos_total_ss # -0.001413857

