3
$\begingroup$

I am trying to model some data, and as part of the modeling, I tried linear model (using lm function) and non-linear model (using nls) function.

  • Model 1: a linear model which has degrees of freedom (df), (AIC) = 2, 2130

  • Model 2: a non-linear model which has df, AIC= 4, 2128.

  • Model 3: I took model 2 above and fixed a parameter to the estimated value in model 2, it resulted in df, AIC= 3, 2126.

My question is – how to select the best fitting model from the above candidate models? Can I simply use AIC (lower is better) and therefore select model 3?

Best regards.

$\endgroup$

3 Answers 3

0
$\begingroup$

The question is indeed rather theoretical. What is the best theoretical model by construction (i.e. how do the variables connect to each other? Is the theoretical dependence of linear or non-linear nature?).

However, if that is not possible, I suggest to compare the models using BIC and AIC as well and select some model inbetween those two. From my personal experience AIC is better for predictions and BIC better for fitting (explanatory).

$\endgroup$
0
$\begingroup$

If you are insterested in predictions: MAPE :mean absolute percentage error (MAPE), is a measure of prediction accuracy of a forecasting method in statistics.

install.packages("MLmetrics") library(MLmetrics) MAPE(predict(model),real_values)

$\endgroup$
0
$\begingroup$

You will find unending debates on the question. There is no good answer on that question. Generally, people tend to prefer BIC rather than AIC. BIC selects more parcimonious models because the penalty for the number of parameters is higher.

If you are more interested in predictions, you might use other performance criterion. Especially root mean square error (RMSE).

I think you will find many interesting debates on this question on CrossValidated.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.