# Query about one step in AR-sieve bootstrap

I am trying to understand how an AR-sieve bootstrap works.

Here are the steps which are to be taken to do a sieve bootstrap:-

1. We are given a sequence $$X_1,..., X_n$$.
2. We compute residuals :

$$\hat{\epsilon}_{t,n} = \sum_{j=0}^{p(n)} \hat{\phi}_{j,n}(\hat{X}_{t-j} - \bar{X}),\quad t=p+1,...,n ,\phi_{0,n}=1$$

1. $$\tilde{\epsilon}_{t,n} = \hat{\epsilon}_{t,n} - (n-p)^{-1} \sum_{t=p+1}^{n} \hat{\epsilon}_{t,n}$$

2. We call the empirical cumulative distribution function of $$\tilde{\epsilon}_{t,n}$$ as $$\hat{F}_{\epsilon,n}$$, i.e.

$$\hat{F}_{\epsilon,n}(\cdot) = (n-p)^{-1} \sum_{t=p+1}^{n}1[\tilde{\epsilon}_{t,n}\leq\cdot]$$

1. We resample from $$\hat{F}_{\epsilon,n}$$ and define $${X^{*}_{t}}$$ by the recursion:

$$\sum_{j=0}^{p(n)} \hat{\phi}_{j,n}({X}_{t-j}^{*} - \bar{X}) =\hat{\epsilon}_{t,n}^{*}, \quad t=p+1,...,n , \hat{\phi}_{0,n}=1$$

I got the above from page 4 on the pdf link I have posted above. (I have tried to make this query self contained).

My query: I do not know why we need step 3.

We do we need to center the residuals obtained in step 2? They are residuals obtained from the following regression:

$$(\hat{X}_{t} - \bar{X}) = \sum_{j=1}^{p(n)} \hat{\phi}_{j,n}(\hat{X}_{t-j} - \bar{X}) + \hat{\epsilon}_{t,n} \quad t=p+1,...,n,$$ since $$\phi_{0,n}=1$$.

The above is a regression where both the dependent and independent variables have been demeaned. The residuals will have mean zero. I think we do not need step 3. Do I misunderstand?

• Just a minor point, you use both $p$ and $p(n)$ for the lag length; the latter notation emphasizes that the lag length is allowed to increase with time series length, but isn't used consistently in the post. Maybe unify. Commented Jun 20 at 9:10
• Dear Christoph, you are right about that. I wish to leave my post as it is, since I am quoting a paper. If I change my post, it will no longer match the steps in the paper. But yes you are correct and also the paper says that p(n) = o(n) which implies that p changes as n becomes larger and larger. Commented Jun 20 at 11:17

My conjecture is that you are only "almost" right, whence step 3 might still make a (small) difference. The reason for my conjecture is that, since the same $$\bar X$$ appears throughout, the paper subtracts the exact same $$\bar X$$ everywhere.

A sufficicent condition for mean zero residuals is that there is a constant in the regression.

The latter, by the Frisch-Waugh-Lovell theorem, is equivalent to demeaning the dependent variable and each regressor separately, see https://stats.stackexchange.com/a/202030/67799.

Now, the regressors (the lags $$X_{t-j}$$, $$j=1,\ldots,p$$) of course have very similar means as their observations largely overlap. Their means are however not exactly identical, as allowing for $$p$$ lags requires discarding the first $$p$$ initial observations, so that the "regressor matrix" in such an AR(p) model would look like $$X=\left( \begin{array}{ccccc} 1&X_p & X_{p-1} & \cdots & X_1 \\ 1&X_{p+1} & X_p & & X_2 \\ \vdots & \vdots &\vdots & & \vdots \\ 1&X_{n-1} & X_{n-2} & \cdots & X_{n-p} \\ \end{array} \right),$$ and each column for the lags will have slightly different means. Hence, running a AR regression with variables that are demeaned with the same overall mean but without a constant cannot be expected to exactly yield zero mean residuals.

Illustration:

n <- 20
x <- rnorm(n)
xm <- x[2:n]-mean(x)
xml <- x[1:(n-1)]-mean(x)
sum(resid(lm(xm~xml-1)))            # same mean: sum of residuals not zero
sum(resid(lm(x[2:n]~x[1:(n-1)])))   # regression with constant: zero sum of residuals

xdm <- x[2:n]-mean(x[2:n])
xdml <- x[1:(n-1)]-mean(x[1:(n-1)])
sum(resid(lm(xdm~xdml-1)))          # separate means: zero sum of residuals


Many thanks for your reply. Like you say, we have used the sample mean to demean the LHS and each column of the RHS.

For example for an AR(2) model we have:

$$\begin{bmatrix} X_3 - \bar{X} \\ X_4 -\bar{X} \\ ... \\ X_n - \bar{X} \end{bmatrix} = \begin{bmatrix} X_2 -\bar{X} & X_1 -\bar{X} \\ X_3 -\bar{X} & X_2 -\bar{X} \\ ... & ... \\ X_{n-1}-\bar{X} & X_{n-2} -\bar{X} \end{bmatrix} \begin{bmatrix} \phi_1 \\ \phi_2\end{bmatrix} + \begin{bmatrix} \epsilon_3 \\ ... \\ \epsilon_n \end{bmatrix}$$

We should have replaced $$\bar{X}$$ with :

$$\Sigma_{i=3}^{n}X_i$$ : For LHS

$$\Sigma_{i=2}^{n-1}X_i$$ : For column 1 of RHS

$$\Sigma_{i=1}^{n-2}X_i$$ : For column 2 of RHS

in order for the residuals to sum to 0 in accordance with the Frisch-Waugh- Lowell theorem.

So, since the residuals do not sum to 0, we have to center them.

That takes us to the question that, when we do the above "replacement of column means with the overall mean" and run the regression with the centered residuals we will have a valid bootstrap sample.

In the paper which I have cited they say that there are some results which state that the bootstrap samples so generated are conditionally stationary. I quote from page 2 of the paper:

However, we take here the point of view of approximating sieves. As with the blockwise bootstrap, this resampling procedure is again nonparametric and, moreover, its bootstrap sample is (conditionally) stationary and does not exhibit additional artefacts of the dependence structure as above.

That I guess is the full reasoning.