1
$\begingroup$

$R^2$ is usually used as a measure to determine a goodness of a fit. It appears to be used often times for linear least square fits, linear regression.

There's another measure which is RSS (residual sum of squares). What's the difference from statistical application aspect? Can $R^2$ be used for a non-linear least square fits/non-linear regression? if not why?

Improve this question
$\endgroup$
4
  • 3
    $\begingroup$ $R^2$ is not a measure for goodness of fit, despite the fact that a lot of people use it as such. $\endgroup$
    – user2974951
    Commented Oct 28, 2021 at 8:34
  • $\begingroup$ What do you mean by "non-linear" least square fit here? Nonlinear in parameters or in variables? $\endgroup$
    – Dayne
    Commented Oct 28, 2021 at 9:06
  • 1
    $\begingroup$ Readers may find themselves interested in this animation someone posted on here. // Possible duplicate $\endgroup$
    – Dave
    Commented Oct 28, 2021 at 10:50
  • $\begingroup$ The mean RSS is aka as the mean squared error (MSE). $R^2$ is just a normalized variant of the MSE. The problem with both $R^2$ and $RSS$, however, is that they are computed by resubstituting the training data into the model and are thus heavily optimistically biased. The name of this forum provides a better method for assessing model quality ;-) $\endgroup$
    – cdalitz
    Commented Oct 28, 2021 at 19:49

1 Answer 1

Reset to default
1
$\begingroup$

$R^2$ is a function of $RSS = \sum (y_i - \hat y_i)^2$.

$$ R^2 = 1- \dfrac{ RSS } { \sum (y_i - \bar y)^2 } $$

What this equation means is that $R^2$ compares the model quality to always predicting the observed mean of $y$, regardless of your predictor variables.

Consequently, $R^2$ and $RSS$ provide the same information when it comes to model comparisons. If you compare two models of the same $y$, the one with lower $RSS$ has higher $R^2$, and this fact does not depend on model linearity.

I dislike $R^2$ for two reasons, both of which I have mentioned on here in the past, perhaps better than I describe here.

https://stats.stackexchange.com/a/539785/247274

  1. When the model is nonlinear, and even sometimes when it is linear, $R^2$ loses its interpretation from OLS of describing the proportion of explained variance.

Nonlinear regression SSE Loss

Why does regularization wreck orthogonality of predictions and residuals in linear regression?

  1. Even if we can interpret $R^2$ as the proportion of variance explained, people seem to want to think in terms of grades in school, where $R^2=0.9$ means an $\text{A}$-grade on our model, while $R^2=0.5$ means and $\text{F}$-grade on our model. Depending on the problem, $R^2=0.5$ might be fantastic performance that we would be very happy to achieve, while $R^2=0.9$ could be rather pedestrian performance.

Why getting very high values for MSE/MAE/MAPE when R2 score is very good

Improve this answer
$\endgroup$
3
  • $\begingroup$ wow! a lot of people out there is using and suggesting that R^2 is the measure for goodness of a fit. So assuming I have a fit that gives RSS (Residuals sum of squares: 0.011345919449258304) and R2 squared: 0.1992834706376369. what's the correct measure to use? it's a non-linear least square (exponential) function. If a model gives a good R2 for a fit in one software, shouldn't that model give same results or nearly close results in all other statistical programming or graph fitting softwares? (R, Python, Prism Graphpad, Origin etc)? $\endgroup$
    – bonCodigo
    Commented Oct 29, 2021 at 23:29
  • $\begingroup$ Are you able to merge this question and answer to one of the original questions stated above? Are you able to notify it to moderators if you are not one of them. $\endgroup$
    – bonCodigo
    Commented Oct 30, 2021 at 3:35
  • $\begingroup$ for e.g. model is something like this Y = a + ((b-a)/(1 + exp(((k1*q* exp(-k2*d*sqrt(x)))/sqrt(x)) with 4 variables... a, b, q, d. $\endgroup$
    – bonCodigo
    Commented Oct 30, 2021 at 3:44

Not the answer you're looking for? Browse other questions tagged or ask your own question.