Box-Jenkins model selection

Question

The Box-Jenkins model selection procedure in time series analysis begins by looking at the autocorrelation and partial autocorrelation functions of the series. These plots can suggest the appropriate $p$ and $q$ in an ARMA$(p,q)$ model. The procedure continues by asking the user to apply the AIC/BIC criteria to select the most parsimonious model among those that produce a model with a white noise error term.

I was wondering how these steps of visual inspection and criterion-based model selection impact the estimated standard errors of the final model. I know that many search procedures in a cross-sectional domain can bias standard errors downward, for example.

On the first step, how does selecting the appropriate number of lags by looking at the data (ACF/PACF) impact the standard errors for time series models?

I would guess that selecting the model based upon AIC/BIC scores would have an impact analogous to that for cross-sectional methods. I actually don't know much about this area either, so any comments would be appreciated on this point as well.

Lastly, if you wrote down the precise criterion used for each step, could you bootstrap the entire process to estimate the standard errors and eliminate these concerns?

Answer 1

Any model selection procedure will affect the standard errors and this is hardly ever accounted for. For example, prediction intervals are computed conditionally on the estimated model and the parameter estimation and model selection are usually ignored.

It should be possible to bootstrap the whole procedure in order to estimate the effect of the model selection process. But remember that time series bootstrapping is trickier than normal bootstrapping because you have to preserve the serial correlation. The block bootstrap is one possible approach although it loses some serial correlation due to the block structure.

Answer 2

In my opinion selecting the appropriate number of lags is no different than selecting the number of input series in a stepwise forward regression procedure. The incremental importance of lags or a specific input series is the basis for the tentative model specification.

Since you have asserted that the acf/pacf is the only basis for Box-Jenkins model selection, let me tell you what some experience has taught me. If a series exhibits an acf that doesn't decay, the Box-Jenkins approach (circa 1965) suggests differencing the data. But if a series has a level shift, like the Nile data, then the "visually apparent" non-stationarity is a symptom of needed structure but differencing is not the remedy. This Nile dataset can be modeled without differencing by simply identifying the need for a level shift first. In a similar vein we are taught using 1960 concepts that if the acf exhibits a seasonal structure (i.e. significant values at lags of s,2s,3s,...) then we should incorporate a seasonal ARIMA component. For discussion purposes, consider a series that is stationary around a mean and at fixed intervals, say every June there is a "high value". This series is properly treated by incorporating an "old-fashioned" dummy series of 0 and 1's (at June) in order to treat the seasonal structure. A seasonal ARIMA model would incorrectly use memory instead of an unspecified but waiting-to-be-found X variable. These two concepts of identifying/incorporating unspecified deterministic structure are direct applications of the work of I. Chang, William Bell, George Tiao, R.Tsay, Chen et al (starting in 1978) under the general concept of Intervention Detection.

Even today some analysts are mindlessly performing memory maximization strategies, calling them Automatic ARIMA, without recognizing that "mindless memory modeling" assumes that deterministic structure such as pulses, level shifts, seasonal pulses and local time trends are non-existent or worse yet play no role in model identification. This is akin to putting one's head in the sand, IMHO.