how to compare linear and nonlinear regression models in goodness of fit?

Question

Is nonlinear regression (always?) better than linear regression? How can I decide which model to use? I have following alternative models in mind:

$G$ is DV, $x$ and $y$ are IVs

1. $G_i = (b_1(x_i^{b2} - y_i^{b2}) + y_i^{b2})^{1/b2}$ (2 parameters)

2. $G_i = (0.5(x_i^{b2} - y_i^{b2}) + y_i^{b2})^{1/b2}$ (1 parameter)

3. $G_i = b_1(x_i - y_i) + y_i$ (1 parameter)

If my goal is to find the best fit, is calculating AIC, BIC, and adjusted r-squared a good way to select the model between linear and nonlinear regressions? And should I still compare the residual plots? Is there any better way to select the best model given these 3 models?

I should mention that my data size is small.

Thanks!

Answer 1

Obviously, nonlinear regression will not always be better than linear regression, because sometimes relationships are linear.

Models with more parameters will produce higher R² values unless the additional predictors are perfectly correlated with previous ones. Taken to the extreme, adding parameters will lead to meaningless models that fit your data perfectly but perform terribly at out-of-sample prediction and in cross-validation. AIC, BIC, and adjusted R² are metrics used to penalise the additional model parameters to achieve a balance between explanatory/predictive power and model complexity. The specific penalties differ, and the most appropriate one is debated; the need for some penalty is universally agreed upon. Since you have a small dataset, these metrics will tend to favour simpler models. With more data, it is possible that more complex models will be favoured.

Examining residual plots is useful to see whether any particular model fit is appropriate. For example, patterns in the residuals can sometimes suggest that a different model is necessary. They can sometimes justify choosing a more complex model even when the metrics favour a simpler one.

Answer 2

What I would do here is compute scatterplots of G against the predicted G for the 3 different models. I might also compute Tukey mean difference plots. And I'd look at plots of the residuals from the three models.

Then I'd make my decision based on what those plots told me. Models can go wrong in various ways; not all are equally important in every situation. E.g. perhaps, for you, an underestimate of G is much worse than an overestimate (or vice versa). Perhaps you can put up with a lot of small errors if you don't get any big ones. Or maybe something else.