regression - does R2 only apply to measure linear regression performance? [duplicate]

Question

Background

According to Wiki: https://en.wikipedia.org/wiki/Coefficient_of_determination, $R^2$ is coefficient of determinant. The definition is $$ R^2 = 1 - \dfrac{SSE}{SST} $$

Since $SSE$ is simply sum of square of residuals, it does not hurt to be used for nonlinear regression. And I have seen a lot of people doing so, although the adjusted version might be difficult to justify in the nonlinear regression case.

Where is the different voices

So the only trick thing is that, for nonlinear regression in general,
$$ SST \neq SSR + SSE $$ so one can say well, the coefficient of determination might be out of the bound of [0,1]. So there is no such definitino. However, I would say does adjusted $R^2$ in linear regression really guarantee the boundness?

So the key part that ensures the above variance equality is that one would need the following $$\sum_{i}(y_i - \hat{y}_i) (\hat{y}_i - \overline{y})^\top = 0$$, where $y$-ish follows the linear regression style so it is a row vector. It can be decomposed into two parts $$ \sum_i (y_i - \hat{y}_i) \hat{y}_i = 0 $$ $$ \sum_i (y_i - \hat{y}_i) \overline{y} = 0 $$ The second one is always true if $\hat{y} = \alpha + g(x;\beta)$, where $g$ is a function parameterized by $\beta$. Then any extreme point along $\alpha$ would satisfy the second one.

The first one is a bit tricky.

However, its meaning is quite simple: simply have error residual uncorrelated in the linear sense with prediction over the training distribution. Careful readers might notice that I miss an expectation, however, remember the second one holds, so it naturally absorbs.

This makes sense for some cases. Imagine if residual is correlated with prediction in the linear sense, then it means there is still space for improvement for a more complicated nonlinear model. Indeed, see equation 21 of the God of SI: Billings's paper. In that sense, a well trained nonlinear model should approximately have the $$SST \approx SSR + SSE$$

Another voices

There is some research paper showing that only looking at $R^2$ for nonlinear models does not always let you pick the best model, which is certainly acceptable. It depends on what you think is true and I don't think unless, for obvious comparison, it is often difficult to discern two models that performs similarly. Personally, I also have some experiences that finding in nonlinear models $R^2$ around 0.90 does not give me much good results due to data imbalance. So I use other metrics to tell the difference jointly.

Question:

Does it make sense? Can $R^2$ be used for nonlinear regression? My preference is yes we can use it but with caution and better have the scatter plot

Answer 1

In my experience $R^{2}$ is not suited to assess non-linear regression models.

The fact that $SST \neq SSR + SSE$ can lead to a number of problems, some of which you have already pointed out and are contained in the research paper you referenced.

While I was working on my thesis, I also tried to evaluate non-linear regression models using $R^{2}$. This resulted in many deeply negative values. Initially, I attributed this to my models being misspecified. However, even as I managed to improve some of the models (according to some other metrics) the $R^{2}$ value did not indicate this. Therefore I could not rely on $R^{2}$ to distinguish models that performed well from ones that were horrid. The paper you referenced goes further than to state that you do not always pick the right model, it goes on to state you will pick the wrong model more often than not.

To answer your question: "Can we use it?". Of course you can use it, many statistical packages will allow you to compute it with ease. However, I think that it will not be very useful.