I am a bit stumped on the behavior of $R^2$ in non-linear models.

Below is some data and two hyperbolic fits. One in which two parameters are estimated (Model $m_1$), and another in which one parameter is fixed at $100$, and only the other parameter is estimated (Model $m_2$). Model $m_1$ has much smaller residual standard error ($8.01$ on $2$ df) than $m_2$ ($13.7$ on $3$ df) and just by looking at it (see graph below), a better fit than $m_2$ (even though $m_1$ itself could be improved). The residual sums of squares as reported by anova() are $128.35$ for $m_1$ and $565.80$ for $m_2$. With one df difference that yields a p-value for the difference in SS residual of $.12$.

The difference in residual sums of squares and the eyeball fit is not surprising.

Yet, the $R^2$ of $m_2$ (the constrained model) is much better ($R^2$ is computed (probably incorrectly) by squaring the correlation of predicted values and observed values of $Y$). $m_1$ has a squared correlation of observed and predicted Y values of $.921$, whereas $m_2$ has $.995$.

Is $R^2$ useful at all for these models? Can one distinguish these models based on $R^2$ (e.g., an exponential model could be a competing model and we may check $R^2$ of that model).

I am curious about this, because in my corner of the literature these types of models are favored or discarded based on $R^2$ evidence and I also don't understand how a constraint can improve $R^2$ in this situation.

test <- data.frame(x=c(1,10,50,100),y=c(57.7,28.0,17.8,14.8))

m1 <- nls(y ~ a / (1+b*x),test,start=list(a=200,b=.07))
m2 <- nls(y ~ 100 / (1+b*x),test,start=list(b=.07))
coeffm1 <- coefficients(m1)
coeffm2 <- coefficients(m2)


test$m1pred <- fitted(m1)
test$m2pred <- fitted(m2)


curve((y=coeffm1["a"] / (1+coeffm1["b"]*x)),add=T)
curve((y=100 / (1+coeffm2["b"]*x)),add=T,lty="dashed")

Solid line is model m1, dashed line is model m2. Eyeball fit of m1 is better than m2.

Solid line is model m1, dashed line is model m2. Eyeball fit of m1 is better than m2.


1 Answer 1


I decided to move my comment to an answer and discuss it

To expand on my points a little:

Your thought that the way you're calculating $R^2$ isn't sensible is right.

A high correlation between residuals and arbitrary fitted values doesn't automatically imply a good fit. Indeed, forget nonlinear regression, and consider linear models.

Imagine you have two linear models. For the first model, the Flying Spaghetti Monster comes to you in a dream, touches you with his noodly appendage and tells you the true parameter values:

$y_i = 9 + 2\,x_i + e_i$, (and that the errors are independent and identically distributed $N(0,\sigma^2)$)

The next day, as you're telling a friend over a pint about your experience, a passing homeopathy salesman suggests that instead

$y_i = -1000 + 7.3\times 10^{-14}\,x_i + e_i$

Imagine that on looking at the data, the dream-values appear to be about right (least squares estimates come out very close to them), and that $\sigma^2$ is estimated to be really tiny, so that the residuals from both the dream-values and the LS fit are minuscule.

Further, the correlation is almost 1.

Now, what's the correlation of the data with the fitted values from the salesman-in-the-pub's model?

It should be low, right? The model is completely wrong! Its predictions are no better than the mean.

... in fact, its correlation is exactly the same; almost 1.

If that was what $R^2$ was, it would be useless as a way of comparing models.

This sounds like it might provide a useful object lesson in exactly why they shouldn't be using that definition of $R^2$ to compare those models.

However, since they have different numbers of parameters, an unadorned residual sum of squares isn't exactly a fair comparison either.

Even a more correct $R^2$ would not necessarily be a good way of comparing nonlinear models; in the nonlinear realm there's often no good reason to consider a constant-mean model, the 'null' situation.

Indeed, even when comparing two linear models, $R^2$ is not necessarily the best way to do it, for example, for similar reasons that the (to my mind) marginally more sensible comparison of residual sum of squares I mentioned earlier should be avoided with models with different numbers of parameters.

I thought that the squared correlation between observed Y and fitted Y is in fact the standard $R^2$ in OLS.

Well, yes, it's true that $R^2$ for a simple ordinary-least-squares model, is the square of the correlation between observed and predicted, but the ability to interpret it the way you want to interpret it is conditioned on it being the result of OLS. If you assert parameter values, for example, you don't change the correlation, but you lose its interpretation as a measure of fit at all.

Coding your example seems to confirm this and also shows that the homeopathy salesman has a lower R2.

The computation in R isn't accurate for the salesman because of round-off error and accumulated numerical error; this is a mathematically exact relationship that we have to take care over doing numerically.

Observe what happens:

x <- runif(100,0,10) 
y <- 9 + 2*x + rnorm(100,0,.005) 
cor(y,-1000+ 7.3e-4*x)
cor(y,-1000+ 7.3e-7*x)
cor(y,-1000+ 7.3e-10*x)
cor(y,-1000+ 7.3e-14*x)
  • 1
    $\begingroup$ @felize2000 see my edit to my answer which explains the problem you encountered (and also illustrates my point). If you prefer, change the example to $7.3\times10^{-7}$ so that you can emulate it in R with more precision. $\endgroup$
    – Glen_b
    Commented May 30, 2013 at 16:28
  • 1
    $\begingroup$ I don't object so strongly to $R^2$ as square of correlation between observed and predicted as a (descriptive) figure of merit for nonlinear models, so long as there is always consultation of the corresponding scatter plot of observed and predicted and (even more importantly) direct examination of data, fitted and residuals. But -- especially if the model fit is not equivalent to minimising that $R^2$ -- it is undoubtedly to be handled with care. $\endgroup$
    – Nick Cox
    Commented May 30, 2013 at 16:30
  • 2
    $\begingroup$ When comparing nonlinear models with different numbers of parameters, that's probably worth a general question on its own. If we ignore the issue with a different number of free parameters, the thing least squares minimizes directly measures one kind of lack of fit (indeed, one you chose by choosing nonlinear least squares). If you had the same number of parameters (and the same $y$ in both of course), it wouldn't be a bad thing to compare. $\endgroup$
    – Glen_b
    Commented May 30, 2013 at 16:40
  • 2
    $\begingroup$ A serious answer on "how to choose" is that often these functions (a) are suggested on scientific grounds or (b) are at least regarded as consistent with what is known scientifically. Limiting behaviour for small and large $x$ is often crucial to model choice (and should often trump goodness or badness of fit). There are programs around that cycle through a large catalogue of functions and come up with best fit, but it's risky to let someone else's program choose a model for you. $\endgroup$
    – Nick Cox
    Commented May 30, 2013 at 16:43
  • 2
    $\begingroup$ All agreed to Nick. In fact, I already used some curve-fitting software (Sigma-plot) that returns tons of fits all in the $R^2$ range of .95 or above. I did not find this very useful. Turns out in this case though, that there is a theoretical debate on whether an exponential, hyperbolic, or hyperbolic with constraint fit is most appropriate. This is in the area of "social discounting". Leonard Green (Wash U) published on this and I often see arguments with regards to $R^2$ and eyeball fit. $\endgroup$ Commented May 30, 2013 at 16:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.