7
$\begingroup$

The query I have relates to calculating the variance of AR(1) processes that are smoothed with a simple moving average. So:

In an AR(1) process of the form:

$$ X_t=c+\varphi X_{t-1}+\varepsilon_t, $$

the variance can be calculated as:

$$ \text{var}(X_t)=E(X^2_t)-\mu^2=\frac{\sigma^2_{\varepsilon}}{1-\varphi^2}, $$

where $\sigma$ is the standard deviation of $\varepsilon$ (a white noise), and $\varphi$ is the variable defining the autocorrelation properties of the AR(1) process.

I would like to be able to calculate the variance of the AR(1) process after smoothing by a simple unweighted moving average for various window sizes. I have thus far looked at the problem analytically with data, and as the window size of the moving average increases the variance obviously falls, and the way in which it falls (i.e. rate and shape) is dependent on $\varphi$. Clearly, when the moving average window is equal to $N$ (the size of the dataset analysed) then the variance is 0.

Ultimately therefore, is there a way of determining the expected variance of the AR(1) in terms of $\varphi$, $N$, and the size of the moving average window?

$\endgroup$
6
$\begingroup$

Let us rewrite $x_t, x_{t-1}, \dots, x_{t-K+1}$ in terms of $x_{t-K}$

$$x_t=c\left(1+\varphi+\dots+\varphi^{K-1}\right)+\varepsilon_t+\varphi\varepsilon_{t-1}+\dots+\varphi^{K-1}\varepsilon_{t-K+1}+\varphi^Kx_{t-K}$$

$$x_{t-1}=c\left(1+\varphi+\dots+\varphi^{K-2}\right)+\varepsilon_{t-1}+\varphi\varepsilon_{t-2}+\dots+\varphi^{K-2}\varepsilon_{t-K+1}+\varphi^{K-1}x_{t-K}$$ $$\dots$$ $$x_{t-K+1}=c+\varphi x_{t-K}+\varepsilon_{t-K+1}$$

Then we denote a moving average process of $x_t$ of window $K$ as $\tilde{x}_{t}^K$ and have the following

$$\tilde{x}_t^K=\frac{1}{K}\left[\sum_{i=0}^{K-1}\left(c(K-i)\varphi^i+\varepsilon_{t-i}\sum_{j=0}^i\varphi^j\right)+x_{t-K}\sum_{i=1}^K\varphi^i\right]$$

$$\begin{align} \operatorname{\mathbb{V}ar}\left(\tilde{x}^K_t\right) &= \frac{1}{K^2}\left[\operatorname{\mathbb{V}ar}\left(\sum_{i=0}^{K-1}\varepsilon_{t-i}\sum_{j=0}^i\varphi^j\right)+\operatorname{\mathbb{V}ar}\left(x_{t-K}\sum_{i=1}^K\varphi^i\right)\right]\\ &= \frac{1}{K^2}\left[\sigma^2_{\varepsilon}\sum^{K-1}_{i=0}\left(\frac{\varphi^{i+1}-1}{\varphi-1}\right)^2+\frac{\sigma^2_{\varepsilon}}{1-\varphi^2}\left(\frac{\varphi^{K+1}-\varphi}{\varphi-1}\right)^2\right]\\ &=\frac{\sigma^2_{\varepsilon}}{K^2}\left[\frac{1}{(\varphi-1)^2}\sum^{K-1}_{i=0}\left(\varphi^{i+1}-1\right)^2+\frac{\varphi^2(\varphi^K-1)^2}{(1-\varphi)^2(1-\varphi^2)}\right] \end{align}$$

and some more algebra leads to..

$$\operatorname{\mathbb{V}ar}\left(\tilde{x}^K_t\right)=\frac{\sigma^2_{\varepsilon}\left(\varphi^2(K\varphi-K+2)-2\varphi^{1+K}(\varphi-1)+K-\varphi(K+2)\right)}{(1+\varphi)(1-\varphi)^4K^2}$$

sigma <- 2.5
phi <- 0.6
K <- 3
const <- 2

set.seed(321)
eps <- rnorm(1e5, sd = sigma)
x <- filter(c(0, const + eps), filter = phi, method = "recursive")

MAvar <- function(phi, sigma, K)
  sigma^2 / (K^2 * (phi + 1) * (1 - phi)^4) * 
  (phi^2 * (K * phi - K + 2) - 2 * phi^(1 + K) * (phi - 1) + K - phi * (K + 2))

library(zoo)
ma <- rollmean(x, K)

var(ma)
# [1] 6.67111
MAvar(phi, sigma, K)
# [1] 6.640625
$\endgroup$
  • $\begingroup$ Thanks! I guess I might be missing something but the code equation for MAvar doesn't seem to match the derived var equation above? Could you give any more info on how the final equation is arrived at? Also, given that when the moving average window is equal to N the var must be zero, does the equation not need to be expressed in terms of N? $\endgroup$ – user29771 Sep 1 '13 at 8:11
  • $\begingroup$ @swisslog, I only changed the order of some terms in MAvar, compared to the last equation, so they should be the same. I have added two more lines to the derivation, the rest is a couple of geometric sums and simplifications. Now regarding N, the thing is that we are looking at a theoretical variance, not empirical one and, since in theory this smoothed AR process never ends, the theoretical variance does not have to be zero as well as it does not depend on N. $\endgroup$ – Julius Vainora Sep 1 '13 at 10:23
3
$\begingroup$

Another way to do it is to calculate directly using the properties of the auto-covariances which are $\gamma (k) = \rho^{k} \gamma (0)$, where $\gamma (0) = \sigma^{2} / (1 - \rho)$ is the variance.

Thus, the average for K periods is given by \begin{align*} \tilde x_{t} &= \frac1K \sum\limits_{t=1}^{K} x_{t} \end{align*} The average mean is $\mathbb E_{t} \tilde x_{t} = \frac1K \sum\limits_{t=1}^{K} \mathbb E_{t} x_{t} = \mu$ and denoting $\hat x_{t} = x_{t} - \mu$ to simpleify notation. The variance is given by. \begin{align*} V ( \tilde x_{t} ) &=\mathbb E_{t} \left[ \left( \frac1K \sum\limits_{t=1}^{K} x_{t} - \tfrac{n}{n} \mu \right)^{2} \right] =\frac1{K^{2}} \mathbb E_{t} \left[ \left( \sum\limits_{t=1}^{K} (x_{t} - \mu) \right)^{2} \right] =\frac1{K^{2}} \mathbb E_{t} \left[ \left( \sum\limits_{t=1}^{K} \hat x_{t} \right)^{2} \right] \end{align*} This square matrix of $N \times N$ elements with the $ij^{th}$ element being $\hat x_{i} \hat x_{j}$ can be written in terms of its diagonal and twice the upper triangular matrix as the upper and lower halves are symmetric. \begin{align*} V ( \tilde x_{t} ) &= \frac1{K^{2}} \mathbb E_{t} \left[ \sum\limits_{i=1}^{K} \hat x_{i}^{2} + \sum\limits_{i=1}^{K-1} \sum\limits_{j=i+1}^{K} 2 \hat x_{i} \hat x_{j} \right] = \frac1{K^{2}} \mathbb E_{t} \left[ \sum\limits_{i=1}^{K} \gamma (0) + 2 \sum\limits_{i=1}^{K-1} \sum\limits_{j=i+1}^{K} \gamma (j-i) \right] \end{align*} which with a reordering of the summation in the last term from $\sum\limits_{i=1}^{K-1} \sum\limits_{j=i+1}^{K} \gamma (j-i)$ to $\sum\limits_{i=1}^{K-1} \sum\limits_{j=1}^{i} \gamma (j)$ and recalling that $\gamma (k) = \rho^{k} \gamma (0)$ then, \begin{align*} & \sum\limits_{i=1}^{K-1} \sum\limits_{j=1}^{i} \gamma (j) = \gamma (0) \sum\limits_{i=1}^{K-1} \sum\limits_{j=1}^{i} \rho^{k} \end{align*} Now, the geometric sum $\sum\limits_{j=1}^{i} \rho^{k} = \rho + \rho^{2} + \ldots + \rho^{i} $ can be simplified to $\sum\limits_{j=1}^{i} \rho^{k} = \tfrac{\rho (1-\rho^{i} ) }{1- \rho}$ which leaves you with \begin{align*} & \sum\limits_{i=1}^{K-1} \sum\limits_{j=1}^{i} \gamma (j) = \frac{\gamma (0) \rho}{1-\rho} \sum\limits_{i=1}^{K-1} (1 - \rho^{i}) \end{align*} And the final sum can be simplified as follows \begin{align*} \sum\limits_{i=1}^{K-1} (1- \rho^{i}) &= ( 1 - \rho ) + ( 1 - \rho^{2} ) + \ldots + ( 1 - \rho^{K-1} ) = (K-1) - ( \rho + \ldots + \rho^{K-1})\\ &= (K-1) - \left( \frac{\rho (1-\rho^{K-1} ) }{1- \rho} \right) \end{align*} Combining back together, we get \begin{align*} V ( \tilde x_{t} ) &= \frac1{K^{2}} \mathbb E_{t} \left[ \sum\limits_{i=1}^{K} \gamma (0) + 2 \sum\limits_{i=1}^{K-1} \sum\limits_{j=1}^{i} \gamma (j) \right] \\ &= \frac1{K^{2}} \left[ K \gamma (0) + 2 \frac{\gamma (0) \rho}{1-\rho} \left( (K-1) - \left( \frac{\rho (1-\rho^{K-1} ) }{1- \rho} \right) \right) \right] \end{align*} or after some algebra\footnote{ Cancelling terms we have \begin{align*} &= \frac{\gamma (0) }{K^{2}} \left[ K + 2 \frac{ \rho}{1-\rho} \left( (K-1) - \left( \frac{\rho (1-\rho^{K-1} ) }{1- \rho} \right) \right) \right] = \frac{ \gamma (0) }{K^{2} (1 - \rho) } \left[ K (1-\rho) + 2 \rho (K-1) - \left( \frac{ 2 \rho^{2} (1-\rho^{K-1} ) }{1- \rho} \right) \right] \\ &= \frac{ \gamma (0) }{K^{2} (1 - \rho) } \left[ K (1 + \rho) - 2 \rho - \left( \frac{ 2 \rho^{2} (1-\rho^{K-1} ) }{1- \rho} \right) \right] = \frac{ \gamma (0) }{K^{2} (1 - \rho)^{2} } \left[ K (1 + \rho)(1-\rho) - 2 \rho (1 - \rho) - 2 \rho^{2} (1-\rho^{K-1} ) \right] \\ &= \frac{ \sigma^{2} }{K^{2} (1 - \rho)^{2} (1-\rho^{2}) } \left[ K (1 - \rho^{2}) - 2 \rho (1-\rho^{K} ) \right]. \end{align*} } \begin{align*} V ( \tilde x_{t} ) &= \frac{ \sigma^{2} }{K^{2} (1 - \rho)^{2} (1-\rho^{2}) } \left[ K (1 - \rho^{2}) - 2 \rho (1-\rho^{K} ) \right] \end{align*}

Using Julius's method above I get exactly the same answer as this as well. Hope it helps.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.