# Box-Jenkins model selection

The Box-Jenkins model selection procedure in time series analysis begins by looking at the autocorrelation and partial autocorrelation functions of the series. These plots can suggest the appropriate $p$ and $q$ in an ARMA$(p,q)$ model. The procedure continues by asking the user to apply the AIC/BIC criteria to select the most parsimonious model among those that produce a model with a white noise error term.

I was wondering how these steps of visual inspection and criterion-based model selection impact the estimated standard errors of the final model. I know that many search procedures in a cross-sectional domain can bias standard errors downward, for example.

On the first step, how does selecting the appropriate number of lags by looking at the data (ACF/PACF) impact the standard errors for time series models?

I would guess that selecting the model based upon AIC/BIC scores would have an impact analogous to that for cross-sectional methods. I actually don't know much about this area either, so any comments would be appreciated on this point as well.

Lastly, if you wrote down the precise criterion used for each step, could you bootstrap the entire process to estimate the standard errors and eliminate these concerns?

• is bias in standard errors (of parameters?) so important in a-theoretic ARMA models? ARMA models ASFAIK are used for short-run forecasts mostly. The problems with parameters interpretation and their properties are less (least?) important. Of course if you are not meaning the characteristics of an innovation process (error term), planing to produce relevant prediction intervals. Jul 5, 2011 at 8:13
• @Dmitrij, There are two main reasons why I am concerned about bias in the standard errors of the coefficients. The first, as you alluded to, is the creation of prediction intervals. The second is testing for structural breaks in the model, a common question that an economist would be interested in answering. The standard errors generated using a selection procedure should be too small, giving prediction intervals that are too narrow and test statistics that are too large. Jul 6, 2011 at 14:04
• but in a-theoretic models (meaning there is no theory, no structure), structural breaks are little to do with the parameters it would be some general tests, regarding the behavior of the residuals of the model. Well in this case unbiased estimates of models parameters are less important, ARMA simply does not have structural models interpretation. Thus parsimonious models are indeed better predictors, since they well balance the usually poor properties of small sample estimators and the accuracy of prediction. Jul 6, 2011 at 14:25
• Note, that even if you know the data generating process that has a lot of parameters, in small samples simpler model will probably do better predictions, but in structural context the parameters of such a model will be very biased (omitted variable bias)! Jul 6, 2011 at 14:30