ACF and PACF are routinely used for approximate identification of a time series model, e.g. as described here. Say, one takes a look at the plots and guesses that it's something like ARMA(2,0,0)(0,0,1), or ARMA(2,0,0)(1,0,1), or ARMA(2,0,0)(0,0,2). Then he/she fits these three models and checks the updated plots and possibly the values of information criteria.

However, the model pool was clearly suggested by the data, so the amount of overfitting is a lot more than it seems. To make it more explicit, imagine that, instead of eyeballing the plots, I fitted all possible model specifications up to 2nd order and the three models above turned out to have the best AICs in that long list.

Did anyone think of a way to adjust for that or testing out of sample is the only way?

  • $\begingroup$ I'm not sure I understood your question so here it is just a comment. There may be more than one model that performs reasonable well for a finite sample of data. Why wouldn't you settle the issue by comparing the mean square error and the AICc of the models or testing the significance of each parameter? $\endgroup$ – javlacalle Sep 30 '14 at 20:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.