Skip to main content
Cross Validated

Return to Answer

added 168 characters in body
Source Link
mkt
mkt
  • 20.4k
  • 11
  • 81
  • 187

Obviously, nonlinear regression will not always be better than linear regression, because sometimes relationships are linear.

Models with more parameters will generally produce higher R2 values unless the additional predictors are perfectly correlated with previous ones. Taken to the extreme, adding parameters will lead to meaningless models that fit your data perfectly but perform terribly at out-of-sample prediction and in cross-validation. AIC, BIC, and adjusted R2 are metrics used to penalise the additional model parameters to achieve a balance between explanatory/predictive power and model complexity. The specific penalties differ, and the most appropriate one is debated; the need for some penalty is universally agreed upon. Since you have a small dataset, these metrics will tend to favour simpler models. With more data, it is possible that more complex models will be favoured.

Examining residual plots is useful to see whether any particular model fit is appropriate. For example, patterns in the residuals can sometimes suggest that a different model is necessary. They can sometimes justify choosing a more complex model even when the metrics favour a simpler one.

Obviously, nonlinear regression will not always be better than linear regression, because sometimes relationships are linear.

Models with more parameters will generally produce higher R2 values. Taken to the extreme, adding parameters will lead to meaningless models that fit your data perfectly but perform terribly at out-of-sample prediction and in cross-validation. AIC, BIC, and adjusted R2 are metrics used to penalise the additional model parameters to achieve a balance between explanatory/predictive power and model complexity. The specific penalties differ, and the most appropriate one is debated; the need for some penalty is universally agreed upon. Since you have a small dataset, these metrics will tend to favour simpler models. With more data, it is possible that more complex models will be favoured.

Examining residual plots is useful to see whether any particular model fit is appropriate. For example, patterns in the residuals can sometimes suggest that a different model is necessary.

Obviously, nonlinear regression will not always be better than linear regression, because sometimes relationships are linear.

Models with more parameters will produce higher R2 values unless the additional predictors are perfectly correlated with previous ones. Taken to the extreme, adding parameters will lead to meaningless models that fit your data perfectly but perform terribly at out-of-sample prediction and in cross-validation. AIC, BIC, and adjusted R2 are metrics used to penalise the additional model parameters to achieve a balance between explanatory/predictive power and model complexity. The specific penalties differ, and the most appropriate one is debated; the need for some penalty is universally agreed upon. Since you have a small dataset, these metrics will tend to favour simpler models. With more data, it is possible that more complex models will be favoured.

Examining residual plots is useful to see whether any particular model fit is appropriate. For example, patterns in the residuals can sometimes suggest that a different model is necessary. They can sometimes justify choosing a more complex model even when the metrics favour a simpler one.

Source Link
mkt
mkt
  • 20.4k
  • 11
  • 81
  • 187

Obviously, nonlinear regression will not always be better than linear regression, because sometimes relationships are linear.

Models with more parameters will generally produce higher R2 values. Taken to the extreme, adding parameters will lead to meaningless models that fit your data perfectly but perform terribly at out-of-sample prediction and in cross-validation. AIC, BIC, and adjusted R2 are metrics used to penalise the additional model parameters to achieve a balance between explanatory/predictive power and model complexity. The specific penalties differ, and the most appropriate one is debated; the need for some penalty is universally agreed upon. Since you have a small dataset, these metrics will tend to favour simpler models. With more data, it is possible that more complex models will be favoured.

Examining residual plots is useful to see whether any particular model fit is appropriate. For example, patterns in the residuals can sometimes suggest that a different model is necessary.