1
$\begingroup$

I have a time series for 1000 days for many firms. I am interested to know, in general, on what basis I should select an ARMA model (the nature of my problem restricts integration order to 0). Should it be based on a predictive measure (e.g. MAE, RMSE) or AIC/BIC (measures of the closeness of a model to the unknown data generation process)?

$\endgroup$
2
$\begingroup$

There is no very strong agreement among forecasters or time series modelers on how to choose a model.

  • If you are more interested in forecasting, I would recommend using a holdout sample: hold back the last, say, 100 data points, fit your models to the first 900 data points, forecast out into your holdout period and examine the quality of this forecast using whatever accuracy measure makes sense in your application. Pick the model that performs best in the holdout sample.

  • If you are more interested in understanding the historical dynamics (maybe you are only interested in detecting change points or such), you could also use information criteria like AIC or BIC.

That said, often forecasters use information criteria, or time series modelers may use holdout samples, arguing that you can only claim you have understood a time series well if you can forecast it well.

You may profit from reading through previous questions tagged both "ARMA" and "model-selection", or both "ARIMA" and "model-selection".

$\endgroup$
2
  • $\begingroup$ Is there any relation between forecasting and historical dynamics. And if so then why can not AIC or RMSE can be used interchangeably for similar model types. $\endgroup$ – Mumbo.Jumbo May 16 '16 at 5:51
  • $\begingroup$ There should be a relationship: if you got historical dynamics very wrong (like overestimating the historical mean by a factor of 10), your forecasts will likely be wrong. However, dynamics change, so the relationship will not be completely straightforward. In-sample RMSE is not useful, because it can tempt you to overfit. $\endgroup$ – Stephan Kolassa May 16 '16 at 8:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.