How do you do bootstrapping with time series data?

Question

I recently learned about using bootstrapping techniques to calculate standard errors and confidence intervals for estimators. What I learned was that if the data is IID, you can treat the sample data as the population, and do sampling with replacement and this will allow you to get multiple simulations of a test statistic.

In the case of time series, you clearly can't do this because autocorrelation is likely to exist. I have a time series and would like to calculate the mean of the data before and after a fixed date. Is there a correct way to do so using a modified version of bootstrapping?

Answer 1

As @cardinal points out, variations on the 'block bootstrap' are a natural approach. Here, depending on the method, you select stretches of the time series, either overlapping or not and of fixed length or random, which can guarantee stationarity in the samples (Politis and Romano, 1991) then stitch them back together to create resampled times series on which you compute your statistic. You can also try to build models of the temporal dependencies, leading to the Markov methods, autoregressive sieves and others. But block bootstrapping is probably the easiest of these methods to implement.

Gonçalves and Politis (2011) is a very short review with references. A book length treatment is Lahiri (2010).

Answer 2

The resampling method introduced in Efron (1979) was designed for i.i.d. univariate data but is easily extended to multivariate data. As discussed in . If $ x_1,···,x_n $ is a sample of vectors, to maintain the covariance structure of the data in the sample. It is not immediately obvious whether one can resample a time series $ x_1,x_2,···,x_n $. A time series is essentially a sample of size 1 from a stochastic process. Resampling a sample is original sample, so one learns nothing by resampling. Therefore, resampling of a time series requires new ideas.

Model-based resampling is easily adopted to time series. The resamples are obtained by simulating the time series model. For example, if the model is ARIMA(p,d,q), then the resamples of an ARIMA(p, q) model with MLEs (from the differenced series) of the autoregressive and moving average coefficients and the noise variance. The resamples are the sequences of partial sum of the simulated ARIMA(p, q) process.

Model-free resampling of time series is accomplished by block resampling, also called block bootstrap, which can be implemented using the tsboot function in R’s boot package. The idea is to break the series into roughly equal-length blocks of consecutive observations, to resample the block with replacement, and then to paste the blocks together. For example, if the time series is of length 200 and one uses 10 blocks of length 20, then the blocks are the first 20 observations, the next 20, and so forth. A possible resample is the fourth block (observation 61 to 80), then the last block (observation 181 to 200), then the second block (observation 21 to 40), then the fourth block again, and so on until there are 10 blocks in the resample.