How to measure the goodness-of-fit of a nonlinear model? Is $R^2$ useful?

Question

Well… I did search for a while before asking and noticed perhaps my question itself has something basically wrong after reading this and this but still not sure so decided to cry out loud :).

As someone unfamiliar with statistics, recently through this post I was somewhat shocked to learn that R-squared value is not a suitable metric for nonlinear models and began to wonder what's proper for measuring goodness-of-fit of a nonlinear model.

Another blog in the previous link suggests using standard error, but it looks like just a personal view. Is there any consensus for this issue? If not, how to decide what metric should be used? Simply throw the specific model to this site and ask?

Answer 1

I disagree with the Minitab blog and many of the common criticisms of $R^2$. After all, $R^2$ is just a function of the sum of squared residuals: $SSRes = \sum_{i=1}^n\big(y_i - \hat y_i\big)^2$.

$$ R^2 = 1-\dfrac{SSRes}{\sum_{i=1}^n\big(y_i - \bar y\big)^2} $$

This equation is equivalent to other common definitions of $R^2$, such the the squared correlation between $X$ and $Y$ in simple linear OLS regression and the squared correlation between true and predicted values in both simple and multiple OLS linear regression.

Consequently, any criticism of $R^2$ is also a criticism of the sum of squared residuals and the mean squared residual, $MSRes = \dfrac{SSRes}{n}$, which often gets called the mean squared error or $MSE$.

A major (and valid) criticism of all of these metrics is that they can be driven to be perfect by overfitting to the data. If we hit every $y_i$ point, then every residual is zero, the $SSRes$ is zero, and the $R^2$ is one.

Consequently, we want to have some way to penalize the model for overfitting. There are two main avenues for doing this: sticking to in-sample (training) data and testing on some holdout (test or validation data) data.

To penalize the model for perhaps overfitting, a common in-sample approach is to tweak $R^2$ and use adjusted $R^2$. $$R^2_{adj}=1 - \dfrac{\dfrac{SSRes}{n-p}} {\dfrac{\sum_{i=1}^n\big(y_i - \bar y\big)^2}{n-1}}$$.

Here, $p$ is the number of parameters in the regression (including the intercept).

An alternative to using an in-sample metric like $R^2_{adj}$ is to keep some data that the model has not seen and test out the model on this holdout data. If the model has overfitted, we expect poor performance on the holdout data. The usual metrics applied to holdout data would work well here. Beyond just considering functions of squared residuals, we might be interested in absolute residuals or percent deviation between truth and prediction.

A more advanced variant of out-of-sample validation, beyond the scope of this question, uses bootstrapping to estimate by how much you have overfit. I briefly describe the method here.

But you had asked about nonlinear models. Analogues to $R^2$ for nonlinear models are hard to determine, as the degrees of freedom used in place of the $n-p$ term are not as clear as in linear regression. Consequently, using out-of-sample checks (or bootstrap validation) might be the way to go. Many in-sample metrics can be calculated on out-of-sample data. I will list a few below along with pros and cons.

$SSRes$

Pros: Easy to calculate

Cons: Hard to interpret, since it can grow large just by having many observations

$MSE$

Pros: Easy to calculate, related to the variance of the error term

Cons: The relationship to variance can range from unhelpful to downright misleading if the error is not Gaussian or does not have a constant variance; the units are squared

$RMSE: \text{Root Mean Squared Error}$

(This is just the square root of the MSE.)

Pros: Related to the standard deviation of the error term; easy to calculate; in the same units of $y$

Cons: The relationship to standard deviation can range from unhelpful to downright misleading if the error is not Gaussian or does not have a constant standard deviation

$R^2$

Pros: Related to comparing your predictions to the predictions of a baseline model

Cons: Out-of-sample (and even in-sample when a regression is fit by a method other than least squares), $R^2$ lacks its usual "proportion of variance explained" interpretation; it is easy to think in term of letter grades in school where $R^2=0.6$ is a $\text{D}$ that makes us sad, even though such a value might be spectacular performance

(For reasons I discuss in detail here, I disagree with the exact implementation of out-of-sample $R^2$ in the common Python machine learning package sklearn. That implementation compares your performance to a model that always guesses the out-of-sample mean, which is supposed to be a model that you cannot access (since the out-of-sample data are not for training).)

$MAPE: \text{Mean Absolute Percentage Error}$

Pros: Handles data on different scales, where missing by $5$ might be a big deal when the true value is $10$ but less of a big deal when the true value is a billion

Cons: Overestimates and underestimates are not penalized equally; you have to divide by zero if a true value is zero; many others, as described on the Wikipedia article on MAPE, though the link also mentions some alternatives

Answer 2

(Major edit, Aug 5, 2022)

The Minitab blog cited by the OP states why Minitab doesn't include $R^2$ with the results of nonlinear regression. They cite Spiess and Neumeyer (1) who evaluate various methods of choosing among two (or more) nonlinear models. Choosing the model that fits with the highest $R^2$ is far from the best method. Note that they compared models with different numbers of parameters, and $R^2$ doesn't "penalize" models with more parameters like AIC and BIC do.

But there are other reasons to evaluate goodness-of-fit besides choosing among models. Sometimes the goal is to compare the fit of one set of data with the fit from prior runs of the same experiment (always fitting the same model). For this purpose, I think $R^2$ is useful.

Say your lab repeats an experiment many times (with some variations of course) so know to always expect $R^2$ values between 0.6 and 0.8. If a new experiment has $R^2$=0.2, you would be suspicious and look carefully to see if something went wrong with the methods or reagents used in that particular experiment. And if a new employee brings you results (using the same experimental system) with $R^2$=0.95, you would be suspicious (too good to be true) and look carefully at how many "outliers" were removed, whether any data was made up, whether the analysis was conducted properly, .... Or maybe this new employee was a more careful experimenter and so obtained cleaner data, and your expectations for $R^2$ need to be updated.

Bottom line. Whether or not $R^2$ is a useful way to assess goodness-of-fit in nonlinear regression depends on why you are assessing goodness-of-fit.

More detail: $R^2$ and sum-of-squares-of-residuals really do measure goodness of fit. AIC, AICc, BIC (etc) assess the tradeoff of goodness-of-fit vs. number of parameters in the model (number of degrees of freedom really). When comparing models, these values are better (more likely to lead to the correct model) because they don't just measure goodness-of-fit but also take into account number of df (which depends on number of parameters fit by the model).

1. Spiess, Andrej-Nikolai, Natalie Neumeyer. An evaluation of R2 as an inadequate measure for nonlinear models in pharmacological and biochemical research: a Monte Carlo approach. BMC Pharmacology. 2010; 10: 6.

Answer 3

Regression has to do with the whole study, the type of data, the correct statistical inference, the correct form, and the right tests just to name a few. In other words, R-square value can be used but not sufficient. This is true even in linear models. What is most important is making sure the theory behind the model is logical since you can have goodness of fit yet still be far off in your theory. Null hypothesis and other metrics should be used as well as R-squared or modified R-square. In other words, there is so much more to regression than testing. Much of it is understanding the statistical inference then applying the correct metrics to the specific data you are using. For example is it cross sectional data? Is the data qualitative? You can spend many a semester in graduate school studying this so what I give you is a mere taste.