# Here's why you can (hopefully) use $R^2$ for non-linear models. Why not?

We calculate $$R^2$$ as follows:

$$R^2 = 1 - \frac{\|y - \hat y\|^2}{\|y - \bar y\|^2}$$

1. $$y$$ is a vector of true answers;
2. $$\bar y$$ is a vector whose elements are mean of $$y$$;
3. $$\hat y$$ is a a vector with our model's predictions.

So, in case of OLS Linear Regression $$R^2 = 1 - \sin^2 \theta = \cos^2 \theta$$, where $$\theta$$ is an angle between vectors $$y - \bar y$$ and $$\hat y - \bar y$$.

Everybody says that it's forbidden to use $$R^2$$ in case of non-linear models. So, I've been pondering and trying to imagine why it is so and still disagree with that. Here is my line of reasoning. Suppose we have some non-linear model and here is all you need to know about it:

You can see from this GIF that $$\hat y - \bar y$$ is not orthogonal to $$y - \hat y$$. So, $$SS_{tot} ≠ SS_{exp} + SS_{res}$$, where $$SS_{tot} = \|y - \bar y\|^2$$, $$SS_{exp} = \|\hat y - \bar y\|^2$$ and $$SS_{res} = \|y - \hat y\|^2$$. It's obvious from the Pythagorean theorem. But why do we need that equality to be true? Look at how we calculate $$R^2$$. We don't actually calculate $$SS_{exp}$$ explicitly. Instead we calculate it as a difference between $$SS_{tot}$$ and $$SS_{res}$$.

Let's look at what's happening when we calculate $$R^2$$ in the case of our non-linear model:

When you calculate $$\|y - \bar y\|^2 - \|y - \hat y\|^2$$ in the numerator according to the Pythagorean theorem it is equivalent to calculating the squared length of the vector $$(\hat y) - \bar y$$. It just means that if your model was linear, then your best fit solution would lie where the green point $$(\hat y)$$ is lying. $$R^2$$ is $$\cos^2 \theta$$. But now $$\theta$$ is no longer the angle between vectors $$y - \bar y$$ and $$\hat y - \bar y$$, but between vectors $$y - \bar y$$ and $$(\hat y) - \bar y$$.

There are actually infinitely many points $$\hat y$$ such that $$\|y - \bar y\|^2 - \|y - \hat y\|^2$$ is equal to $$\|(\hat y) - \bar y\|^2$$. For every red point that's true:

From here we can't say that $$\left((\hat y) - \bar y\right) + \left(y - \hat y\right) = \left(y - \bar y\right)$$, but the variance of $$y$$ decreased exactly by $$\|(\hat y) - \bar y\|^2$$, because $$\|y - \bar y\|^2 - \|(\hat y) - \bar y\|^2 = \|y - (\hat y)\| = \|y - \hat y\|^2$$. And the closer the red point $$\hat y$$ is to $$y$$, the smaller is the value of $$R^2 = \cos^2 \theta$$.

This is as much meaningful as it is in case of OLS Linear Regression. So, if everything I said was right, then why can't we use $$\mathbf{R^2}$$ for non-linear models? If $$R^2 = 0.86$$ then your model's variance decreased by $$86\text%$$ (no matter linear or not).

• "Everybody says that it's forbidden to use $R^2$ in case of non-linear models." This is just not true. There is a serious case that the square of the correlation between observed and predicted values is something that can generally be calculated. How useful it is, how closely related it is to anything else, and how far experiences with linear regression carry over to other cases are all detailed questions. Oct 11, 2021 at 16:07
• Cox, D. R. and N. Wermuth. 1992. A comment on the coefficient of determination for binary responses. American Statistician 46: 1–4. is a paper warning you to be careful. . Zheng, B. and A. Agresti. 2000. Summarizing the predictive power of a generalized linear model. Statistics in Medicine 19: 1771–1781. is positive about wider use, with cautions. Oct 11, 2021 at 16:12
• @Nick Cox, Isn't it as much meaningful as in case of OLS Linear Regression? In both cases we learn by how much your model's variance decreased w.r.t its initial (total) variance. Oct 11, 2021 at 16:16
• For a given problem, where $y$ is fixed (and we are exploring different models), I find the algebraically equivalent value $||y - \hat y||^2$ (or its square root) to be more useful, because it directly expresses a measure of discrepancy. It is a short step from this to evaluating AIC or BIC (when errors are assumed to be Normally distributed). Thus, any criticism of $R^2$ indirectly attaches to these common applications. BTW, please use animations judiciously. Unless they are the only way you can communicate a critical idea, they detract so much from the text they make it almost unreadable.
– whuber
Oct 11, 2021 at 16:16
• The animation is clever but doesn't really help (me at all). Oct 11, 2021 at 16:22

We can use $$R^2$$ for nonlinear models. In a model comparison, higher $$R^2$$ means lower $$MSE$$, even if the models are nonlinear. However, because of the lack of orthogonality, $$R^2$$ loses its interpretation of proportion of variance explained, so its use as an absolute measure of model performance (“Sure, model A beats model B, but is model A any good?”) is limited.

(For $$MSE$$, I assume that we divide by $$n$$ or $$n-1$$, not $$n-p$$.)

• Thank you for your reply! But why do you say that $R^2$ loses its interpretation of proportion of variance explained? In both cases (I mean linear and non-linear models) we learn by how much your model's variance decreased w.r.t its initial (total) variance. So, why can't you say whether the model A is any good? If $R^2 = 0.86$ then your model's variance decreased by $86\text%$ (no matter linear or not). Oct 11, 2021 at 17:10
• @mathgeek When you decompose the total sum of squares, you wind up with the sum of squares of the regression, the sum of squares of the residuals, and an "other" term. In OLS linear regression (but not necessarily every other linear regression approach), that "other" term is zero, meaning that $R^2 =1- \dfrac{SSRes}{SSTotal}$ is the proportion of variability explained by the model.
– Dave
Nov 1, 2021 at 14:45
• Doesn't "explained" mean that variance decreased? You can actually calculate R-squared in two ways, $R^2 = \frac{SS_{tot} - SS_{res}}{SS_{tot}}$ and $R^2 = \frac{SS_{exp}}{SS_{tot}}$. Yes, in both cases $SS_{exp} ≠ SS_{tot} - SS_{res}$ for non-linear models, but why do you even need that equality? I mean you always calculate R-squared as $R^2 = \frac{SS_{tot} - SS_{res}}{SS_{tot}}$ and in this case it behaves in the same way for both linear and non-linear models. I just can't figure out why we want $SS_{exp} = SS_{tot} - SS_{res}$ to be true, if we never calculate $SS_{exp}$ implicitly. Nov 1, 2021 at 15:05
• Why do we want that "other" term to be zero, if the only thing we end up with is "how much variance is decreased"? And for both linear and non-linear models R-squared does its job telling us this value Nov 1, 2021 at 15:06
• You're always allowed to calculate $R^2 = 1 - \dfrac{SSRes}{SSTotal}$, but without that "other" term being zero, the connection to "proportion of variance explained" is lost. // I do not follow what you mean by "why do we want the other term to be zero?" Could you please elaborate?
– Dave
Nov 1, 2021 at 15:07