According to Wiki: https://en.wikipedia.org/wiki/Coefficient_of_determination, $R^2$ is coefficient of determinant. The definition is $$ R^2 = 1 - \dfrac{SSE}{SST} $$

Since $SSE$ is simply sum of square of residuals, it does not hurt to be used for nonlinear regression. And I have seen a lot of people doing so, although the adjusted version might be difficult to justify in the nonlinear regression case.

Where is the different voices

So the only trick thing is that, for nonlinear regression in general,
$$ SST \neq SSR + SSE $$ so one can say well, the coefficient of determination might be out of the bound of [0,1]. So there is no such definitino. However, I would say does adjusted $R^2$ in linear regression really guarantee the boundness?

So the key part that ensures the above variance equality is that one would need the following $$\sum_{i}(y_i - \hat{y}_i) (\hat{y}_i - \overline{y})^\top = 0$$, where $y$-ish follows the linear regression style so it is a row vector. It can be decomposed into two parts $$ \sum_i (y_i - \hat{y}_i) \hat{y}_i = 0 $$ $$ \sum_i (y_i - \hat{y}_i) \overline{y} = 0 $$ The second one is always true if $\hat{y} = \alpha + g(x;\beta)$, where $g$ is a function parameterized by $\beta$. Then any extreme point along $\alpha$ would satisfy the second one.

The first one is a bit tricky.

However, its meaning is quite simple: simply have error residual uncorrelated in the linear sense with prediction over the training distribution. Careful readers might notice that I miss an expectation, however, remember the second one holds, so it naturally absorbs.

This makes sense for some cases. Imagine if residual is correlated with prediction in the linear sense, then it means there is still space for improvement for a more complicated nonlinear model. Indeed, see equation 21 of the God of SI: Billings's paper. In that sense, a well trained nonlinear model should approximately have the $$SST \approx SSR + SSE$$

Another voices

There is some research paper showing that only looking at $R^2$ for nonlinear models does not always let you pick the best model, which is certainly acceptable. It depends on what you think is true and I don't think unless, for obvious comparison, it is often difficult to discern two models that performs similarly. Personally, I also have some experiences that finding in nonlinear models $R^2$ around 0.90 does not give me much good results due to data imbalance. So I use other metrics to tell the difference jointly.


Does it make sense? Can $R^2$ be used for nonlinear regression? My preference is yes we can use it but with caution and better have the scatter plot

  • 2
    $\begingroup$ I calculate R-squared as "1.0 - (absolute_error_variance / dependent_data_variance)" and use it to tell me what fraction of the dependent data variance is explained by the model. If the R-squared is 0.95, I interpret this to mean that the model explains 95% of the variance in the dependent data. I understand that R-squared is exact for straight line models and approximate for other models, and with this caveat in mind I understand it to be a widely used fit statistic for both linear and nonlinear modeling. $\endgroup$ Sep 27, 2018 at 13:17
  • $\begingroup$ @JamesPhillips Yes. There is a choice to be made since $SST \neq SSR + SSE$. Your choice is the same made by Scikit-learn Python Packages. $\endgroup$ Sep 27, 2018 at 17:55

1 Answer 1


In my experience $R^{2}$ is not suited to assess non-linear regression models.

The fact that $SST \neq SSR + SSE$ can lead to a number of problems, some of which you have already pointed out and are contained in the research paper you referenced.

While I was working on my thesis, I also tried to evaluate non-linear regression models using $R^{2}$. This resulted in many deeply negative values. Initially, I attributed this to my models being misspecified. However, even as I managed to improve some of the models (according to some other metrics) the $R^{2}$ value did not indicate this. Therefore I could not rely on $R^{2}$ to distinguish models that performed well from ones that were horrid. The paper you referenced goes further than to state that you do not always pick the right model, it goes on to state you will pick the wrong model more often than not.

To answer your question: "Can we use it?". Of course you can use it, many statistical packages will allow you to compute it with ease. However, I think that it will not be very useful.

  • $\begingroup$ Thank you for your comment. Can you share the other metrics that you are using? Say standard error of regression? RMSE? $\endgroup$ Sep 27, 2018 at 8:24
  • 1
    $\begingroup$ I don't have much metrics to add to that: This source advises to use the standard error of the regression. Although using RMSE or MSE should do as well, as the idea is similar. Better models should lead to smaller errors. $\endgroup$ Sep 27, 2018 at 8:38
  • 2
    $\begingroup$ Unlike RMSE and MSE, the R-squared value has no units. This is one of the reasons it is both widely used and widely understood, giving the same understanding of model performance whether the units are micro-liters or light-years with the caveat that it is approximate for models other than straight lines. $\endgroup$ Sep 27, 2018 at 15:01
  • 2
    $\begingroup$ I have a hard time accepting the initial statement about "not suited," because when the loss function is quadratic, then SSE or (equivalently) $R^2$ is exactly the right measure to be evaluating! Thus, my interpretation of the rest of your post is that your experience comprises situations with non-quadratic loss functions. In other words, you are stating $R^2$ is not suited because of how you implicitly measure model performance. That looks like a key point to emphasize. $\endgroup$
    – whuber
    Sep 27, 2018 at 15:20
  • 1
    $\begingroup$ If we take $R^2=1-\frac{SSE}{SST}$, as usual, I have to believe that minimizing $SSE$ is the same as maximizing this$R^2$. Maximizing the proportion of variance explained by the model, though, would be a different story. $\endgroup$
    – Dave
    Jul 5, 2020 at 11:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.