2
$\begingroup$

Consider a random variable following an AR(1) model:

$$x_t = \mu+\rho x_{t-1} + \epsilon_t$$

Assumme that $\epsilon_t$ follows $N(0,\sigma^2_\epsilon)$. Now consider the rolling variance and/or standard deviation of this process based on a rolling window of n periods:

$$x_n^{var}=\frac{1}{n-1}\sum_{k=0}^n(x_k-\bar{x}_n)^2$$

Where $\bar{x_t}$ also is the rolling average on window of n periods. I am interested in the expected value of $x_t^{var}$. This is what I have so far. First, use that $x_t$ can be written based on $x_{t-n}$: $$x_t = \mu \sum_{i=0}^{n-1} \rho^i + \sum_{i=0}^{n-1} \rho^i \epsilon_{t-i} + \rho^n x_{t-n}$$

Then inserting this into $\bar{x}_t$ yields: $$\bar{x}_n=1/n\sum_{i=0}^{n-1}(\rho^{n-i}x_{t-n}+\mu\sum_{j=0}^{n-i-1}\rho^j+\sum_{j=0}^{n-i-1}\rho^{j}\epsilon_{t-i-j})$$

$$=1/n(\sum_{i=0}^{n-1}x_{t_n}\rho^{n-i}+ \mu\sum_{i=0}^{n-1}\sum_{j=0}^{n-i-1}\rho^{j}+\sum_{i=0}^{n-1}\sum_{j=0}^{n-i-1}\rho^j\epsilon_{t-i-j})$$

Notice, this can also be stated as: $$\bar{x}_n=x_{t-n}\sum_{i=1}^{n}\rho^i+\sum_{i=0}^{n-1}(\mu(n-i)\rho^i+\epsilon_{t-i}\sum_{j=0}^i\rho^j)$$

Here is, what I have done so far:

  1. Find an expression for the inner of the summand of $x^{var}_t$: $$(x_k-\bar{x}_n)^2 = [x_{t-n}(\rho^k-\frac{1}{n}\sum_{i=1}^{n}\rho^i)+\mu(\sum_{i=0}^{k-1}\rho^k-\frac{1}{n}\sum_{i=0}^{n-1}\mu(n-i)\rho^i)+\sum_{i=0}^{k-1}\epsilon_{t-k+i+1}\rho^{i}-\sum_{i=1}^{n}\epsilon_{t-i}\sum_{j=0}^i\rho^j]^2$$
  2. Tried to reduce the above
  3. Inserted into the summand of $x_{n}^{var}$
  4. Taken expectation

However, I have made errors, as I get unreasonable values, e.g. negative variance... :D. Could someone help derive: $$E[x_{n}^{var}]$$

All help is appreciated

$\endgroup$
7
  • $\begingroup$ Rather than handling sums, I would use vectors and matrices and set $\mu= 0$ (no loss of generality). For a given $n$ the variance writes as $\mathbf{x}^\top \mathbf{Q}\mathbf{x}$ where $\mathbf{x} := [x_1, \, \dots,\, x_n]^\top$ and $\mathbf{Q}$ is a square matrix. Also $\boldsymbol{\varepsilon} = \mathbf{L} \mathbf{x}$ where both matrices $\mathbf{Q}$ and $\mathbf{L}$ are known. $\endgroup$
    – Yves
    Mar 2, 2021 at 5:22
  • 1
    $\begingroup$ There seem to be a problem in the notations with a confusion of indices $n$ and $t$. I guess that the variance is $x^{\text{var}}_n$ and that the average in it is $\bar{x}_n$. $\endgroup$
    – Yves
    Mar 2, 2021 at 6:19
  • $\begingroup$ Hi @Yves, thank you for your comments. I used summations instead of matrix for my own clarity of which time indices were included (to check for covariances). When I use the subscript t on xbar it is to clarify, that this variable is also random over time. But perhaps a x bar with subscript t and a suffix n could suffice? $\endgroup$
    – RVA92
    Mar 2, 2021 at 7:18
  • $\begingroup$ @Yves and feel free to give it a go with the matrix notation, perhaps I can get me in the right direction in comparison to my current apporach :) $\endgroup$
    – RVA92
    Mar 2, 2021 at 7:22
  • 1
    $\begingroup$ In the definition of $x^{\text{var}}_t$ the r.h.s. depends on $n$. So the variance depends on $n$ and $t$? $\endgroup$
    – Yves
    Mar 2, 2021 at 7:32

1 Answer 1

1
$\begingroup$

I will use vector and matrices rather than summations because this makes the results easier to check e.g. using R. Let $\mathbf{x} := [x_1, \, \dots, \, x_n]^\top$ and $\boldsymbol{\varepsilon} := [\varepsilon_1, \, \dots, \, \varepsilon_n]^\top$. I will assume that $\mu = 0$, with no loss of generality because we focus on an expectation. I will denote the variance as $v := x^{\text{var}}_n$ to simplify.

It is easy to see that $v = (n-1)^{-1} \mathbf{x}^\top \mathbf{Q} \mathbf{x}$ where the $n \times n$ matrix $\mathbf{Q}$ is defined by $\mathbf{Q} := \mathbf{I} - n^{-1} \, \mathbf{J}$ where $\mathbf{I}$ is the identity matrix and $\mathbf{J}$ is the matrix of ones. Indeed, subtracting its mean to $\mathbf{x}$ results in $\mathbf{z} = \mathbf{x} - n^{-1} [\mathbf{1}^\top \mathbf{x}] \mathbf{1} = \mathbf{Q}\mathbf{x}$. Note that $\mathbf{Q}$ is symmetric and $\mathbf{Q}^2 = \mathbf{Q}$.

I will assume that the AR starts with $x_1$ having the variance $\sigma_\varepsilon^2$ which is not the stationary initial condition (this will be further discussed later). Then $\boldsymbol{\varepsilon} = \mathbf{L} \mathbf{x}$ where $\mathbf{L}$ and its inverse are $$ \mathbf{L} = \begin{bmatrix} 1 & & & & \\ -\rho & 1 & & & \\ & \ddots & \ddots & & \\ & & & & \\ & & & -\rho & 1 \end{bmatrix} \qquad \mathbf{L}^{-1} = \begin{bmatrix} 1 & & & & \\ \rho & 1 & & & \\ \rho^2 &\ddots & \ddots & & \\ & & & & \\ \rho^{n-1} & & & \rho & 1 \end{bmatrix} $$ where the non-displayed elements are zeros. So $\mathbf{L}$ and $\mathbf{L}^{-1}$ are lower triangular Toeplitz matrices. Now $$ v = \frac{1}{n-1} \, \mathbf{x}^\top \mathbf{Q} \mathbf{x} = \frac{1}{n-1} \, \varepsilon^\top \mathbf{L}^{-\top} \mathbf{Q} \mathbf{L}^{-1} \boldsymbol{\varepsilon}. $$
But if $\mathbf{A}$ is a square matrix we have $\mathbb{E}[\boldsymbol{\varepsilon}^\top \mathbf{A} \boldsymbol{\varepsilon}] = \sigma^2_\varepsilon \, \text{tr}(\mathbf{A})$. The trace being linear with the property $\text{tr}(\mathbf{A}\mathbf{B}) = \text{tr}(\mathbf{B}\mathbf{A})$ we get with $\mathbf{M} := \mathbf{L}^{-1} \mathbf{L}^{-\top}$ $$ \mathbb{E}[v] = \frac{\sigma^2_\varepsilon}{n-1} \, \text{tr}(\mathbf{Q} \mathbf{L}^{-1} \mathbf{L}^{-\top}) = \frac{\sigma^2_\varepsilon}{n-1} \, \left\{ \text{tr}(\mathbf{M}) - \frac{1}{n} \, \text{tr}(\mathbf{J}\mathbf{M}) \right\} $$ It is easy to see that the diagonal element of $\mathbf{M}$ is $M_{ii} = 1 + \rho^2 + \dots + \rho^{2i}$ and with simple algebra $$ \text{tr}(\mathbf{M}) = \sum_{i=1}^n \sum_{j=0}^i \rho^{2j} = \frac{n}{1 - \rho^2} - \frac{\rho^2 \, [1 - \rho^{2n}]}{[1- \rho^2]^2}. $$ Now $\text{tr}(\mathbf{J}\mathbf{M})= \mathbf{1}^\top \mathbf{M} \mathbf{1} = \mathbf{u}^\top \mathbf{u}$ where $\mathbf{u} := \mathbf{L^{-\top}} \mathbf{1}$ with element $u_i = \sum_{j=0}^{n - i} \rho^j = [1 - \rho^{n -i +1}] / [1 - \rho]$. Using $j = n + 1 - i$ as summation index $$ \text{tr}(\mathbf{J}\mathbf{M}) = \sum_{j=1}^n \left[\frac{1 - \rho^j}{1 - \rho}\right]^2 = \frac{1}{[1 - \rho]^2} \left\{ n - 2 \rho \frac{1 - \rho^n}{1 - \rho} + \rho^2 \frac{1 - \rho^{2 n}}{1 - \rho^2} \right\}. $$ So we have a closed form expression for $\mathbb{E}[v]$. For large $n$ we have $\mathbb{E}[v] \sim \sigma^2_\varepsilon / [1 - \rho^2]$

If we turn to the case of a stationary AR we have to take $x_1$ as having the stationary variance $\sigma^2_\varepsilon / [1 - \rho^2]$ so we have to change the element $L_{1,1}$. I fear that the closed form expression will be even more complex, yet the asymptotic behaviour for large $n$ will remain unchanged.

$\endgroup$
6
  • $\begingroup$ Thank you for your excellent answer. I did a simulation study which showed somewhat different values for using the rolling std formula and the closed form from above. How much impact would you believe that the assumption on x1 has? $\endgroup$
    – RVA92
    Mar 2, 2021 at 10:39
  • $\begingroup$ To elaborate: My simulation shows that the two are close, but there are some difference which is diminishing with rho going towards 1 $\endgroup$
    – RVA92
    Mar 2, 2021 at 11:19
  • $\begingroup$ I checked the expressions for the two traces but not the result itself by simulation, which is a good idea. The variance choice for x1 should have an impact for $n$ small or $\rho$ close to $1$. What initial condition did you use? $\endgroup$
    – Yves
    Mar 2, 2021 at 11:31
  • $\begingroup$ As initial condition I used N(mu/(1-rho), sigma^2/(1-rho^2)). As a side question, how do you write formula expressions in comments? $\endgroup$
    – RVA92
    Mar 2, 2021 at 12:01
  • $\begingroup$ The initial condition may explain the differences. I have the good results when initializing as in my answer. Formula in comments are as usual , as you did in the OP e.g. $\mu$ . $\endgroup$
    – Yves
    Mar 2, 2021 at 12:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.