5
$\begingroup$

I am using the forecast package and the auto.arima function. This function tries different arima model with different p and q parameters and selects the best one by AIC.

I tried increasing the default values, so auto.arima searches more possible models and indeed the AIC gets lower. I also did a quick test by splitting the data into a train and test part and the MAPE did also improve slightly.\

However, I am unsure if I could run into overfitting issues when I increase the parameter range further.

$\endgroup$
1
  • 2
    $\begingroup$ Why couldn't you? $\endgroup$
    – Tim
    Aug 3, 2018 at 7:33

2 Answers 2

4
$\begingroup$

It is not very surprising that you get lower AIC when increasing the range, overfitting is always an issue when you try multiple models. AIC or measuring MAPE on the test set helps reduce the risk but once you consider too many models they can mislead you as well.

$\endgroup$
0
$\begingroup$

You have a supervised model, you always have the risk of overfitting or high model variance. You should test for instance, how perturbation affects the model. Does removing one point completely change the model?

If you add more models, there is a higher chance that one of does will do better in that specific training sample, but that does not mean that that model will do better in general. Does removing a point completely change the estimates? Cross validation is a good approach, which is asymptotically related to AIC.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.