Expected value of rolling variance/standard deviation of an AR(1) process

Question

Consider a random variable following an AR(1) model:

$$x_t = \mu+\rho x_{t-1} + \epsilon_t$$

Assumme that $\epsilon_t$ follows $N(0,\sigma^2_\epsilon)$. Now consider the rolling variance and/or standard deviation of this process based on a rolling window of n periods:

$$x_n^{var}=\frac{1}{n-1}\sum_{k=0}^n(x_k-\bar{x}_n)^2$$

Where $\bar{x_t}$ also is the rolling average on window of n periods. I am interested in the expected value of $x_t^{var}$. This is what I have so far. First, use that $x_t$ can be written based on $x_{t-n}$: $$x_t = \mu \sum_{i=0}^{n-1} \rho^i + \sum_{i=0}^{n-1} \rho^i \epsilon_{t-i} + \rho^n x_{t-n}$$

Then inserting this into $\bar{x}_t$ yields: $$\bar{x}_n=1/n\sum_{i=0}^{n-1}(\rho^{n-i}x_{t-n}+\mu\sum_{j=0}^{n-i-1}\rho^j+\sum_{j=0}^{n-i-1}\rho^{j}\epsilon_{t-i-j})$$

$$=1/n(\sum_{i=0}^{n-1}x_{t_n}\rho^{n-i}+ \mu\sum_{i=0}^{n-1}\sum_{j=0}^{n-i-1}\rho^{j}+\sum_{i=0}^{n-1}\sum_{j=0}^{n-i-1}\rho^j\epsilon_{t-i-j})$$

Notice, this can also be stated as: $$\bar{x}_n=x_{t-n}\sum_{i=1}^{n}\rho^i+\sum_{i=0}^{n-1}(\mu(n-i)\rho^i+\epsilon_{t-i}\sum_{j=0}^i\rho^j)$$

Here is, what I have done so far:

1. Find an expression for the inner of the summand of $x^{var}_t$: $$(x_k-\bar{x}_n)^2 = [x_{t-n}(\rho^k-\frac{1}{n}\sum_{i=1}^{n}\rho^i)+\mu(\sum_{i=0}^{k-1}\rho^k-\frac{1}{n}\sum_{i=0}^{n-1}\mu(n-i)\rho^i)+\sum_{i=0}^{k-1}\epsilon_{t-k+i+1}\rho^{i}-\sum_{i=1}^{n}\epsilon_{t-i}\sum_{j=0}^i\rho^j]^2$$
2. Tried to reduce the above
3. Inserted into the summand of $x_{n}^{var}$
4. Taken expectation

However, I have made errors, as I get unreasonable values, e.g. negative variance... :D. Could someone help derive: $$E[x_{n}^{var}]$$

All help is appreciated

Answer 1

I will use vector and matrices rather than summations because this makes the results easier to check e.g. using R. Let $\mathbf{x} := [x_1, \, \dots, \, x_n]^\top$ and $\boldsymbol{\varepsilon} := [\varepsilon_1, \, \dots, \, \varepsilon_n]^\top$. I will assume that $\mu = 0$, with no loss of generality because we focus on an expectation. I will denote the variance as $v := x^{\text{var}}_n$ to simplify.

It is easy to see that $v = (n-1)^{-1} \mathbf{x}^\top \mathbf{Q} \mathbf{x}$ where the $n \times n$ matrix $\mathbf{Q}$ is defined by $\mathbf{Q} := \mathbf{I} - n^{-1} \, \mathbf{J}$ where $\mathbf{I}$ is the identity matrix and $\mathbf{J}$ is the matrix of ones. Indeed, subtracting its mean to $\mathbf{x}$ results in $\mathbf{z} = \mathbf{x} - n^{-1} [\mathbf{1}^\top \mathbf{x}] \mathbf{1} = \mathbf{Q}\mathbf{x}$. Note that $\mathbf{Q}$ is symmetric and $\mathbf{Q}^2 = \mathbf{Q}$.

I will assume that the AR starts with $x_1$ having the variance $\sigma_\varepsilon^2$ which is not the stationary initial condition (this will be further discussed later). Then $\boldsymbol{\varepsilon} = \mathbf{L} \mathbf{x}$ where $\mathbf{L}$ and its inverse are $$ \mathbf{L} = \begin{bmatrix} 1 & & & & \\ -\rho & 1 & & & \\ & \ddots & \ddots & & \\ & & & & \\ & & & -\rho & 1 \end{bmatrix} \qquad \mathbf{L}^{-1} = \begin{bmatrix} 1 & & & & \\ \rho & 1 & & & \\ \rho^2 &\ddots & \ddots & & \\ & & & & \\ \rho^{n-1} & & & \rho & 1 \end{bmatrix} $$ where the non-displayed elements are zeros. So $\mathbf{L}$ and $\mathbf{L}^{-1}$ are lower triangular Toeplitz matrices. Now $$ v = \frac{1}{n-1} \, \mathbf{x}^\top \mathbf{Q} \mathbf{x} = \frac{1}{n-1} \, \varepsilon^\top \mathbf{L}^{-\top} \mathbf{Q} \mathbf{L}^{-1} \boldsymbol{\varepsilon}. $$
But if $\mathbf{A}$ is a square matrix we have $\mathbb{E}[\boldsymbol{\varepsilon}^\top \mathbf{A} \boldsymbol{\varepsilon}] = \sigma^2_\varepsilon \, \text{tr}(\mathbf{A})$. The trace being linear with the property $\text{tr}(\mathbf{A}\mathbf{B}) = \text{tr}(\mathbf{B}\mathbf{A})$ we get with $\mathbf{M} := \mathbf{L}^{-1} \mathbf{L}^{-\top}$ $$ \mathbb{E}[v] = \frac{\sigma^2_\varepsilon}{n-1} \, \text{tr}(\mathbf{Q} \mathbf{L}^{-1} \mathbf{L}^{-\top}) = \frac{\sigma^2_\varepsilon}{n-1} \, \left\{ \text{tr}(\mathbf{M}) - \frac{1}{n} \, \text{tr}(\mathbf{J}\mathbf{M}) \right\} $$ It is easy to see that the diagonal element of $\mathbf{M}$ is $M_{ii} = 1 + \rho^2 + \dots + \rho^{2i}$ and with simple algebra $$ \text{tr}(\mathbf{M}) = \sum_{i=1}^n \sum_{j=0}^i \rho^{2j} = \frac{n}{1 - \rho^2} - \frac{\rho^2 \, [1 - \rho^{2n}]}{[1- \rho^2]^2}. $$ Now $\text{tr}(\mathbf{J}\mathbf{M})= \mathbf{1}^\top \mathbf{M} \mathbf{1} = \mathbf{u}^\top \mathbf{u}$ where $\mathbf{u} := \mathbf{L^{-\top}} \mathbf{1}$ with element $u_i = \sum_{j=0}^{n - i} \rho^j = [1 - \rho^{n -i +1}] / [1 - \rho]$. Using $j = n + 1 - i$ as summation index $$ \text{tr}(\mathbf{J}\mathbf{M}) = \sum_{j=1}^n \left[\frac{1 - \rho^j}{1 - \rho}\right]^2 = \frac{1}{[1 - \rho]^2} \left\{ n - 2 \rho \frac{1 - \rho^n}{1 - \rho} + \rho^2 \frac{1 - \rho^{2 n}}{1 - \rho^2} \right\}. $$ So we have a closed form expression for $\mathbb{E}[v]$. For large $n$ we have $\mathbb{E}[v] \sim \sigma^2_\varepsilon / [1 - \rho^2]$

If we turn to the case of a stationary AR we have to take $x_1$ as having the stationary variance $\sigma^2_\varepsilon / [1 - \rho^2]$ so we have to change the element $L_{1,1}$. I fear that the closed form expression will be even more complex, yet the asymptotic behaviour for large $n$ will remain unchanged.