3
$\begingroup$

Wikepedia, at Variance of Autoregressive model, mentions an expression of variance for an AR(1) process. I am learning signal processing (beginner level) and facing difficulty in understanding some basic relations. It shall be really helpful if the following doubts are answered.

An AR(1) process is defined by $$X_t = c+\theta X_{t-1} + \epsilon_t$$

where $\epsilon_t$ is a zero mean white Gaussian noise. We may compute expected values

$$E[X_t] = E[c] + \theta E[X_{t-1}]+E[\epsilon_t]$$

implying $E[X_t] = \mu = c/(1-\theta)$.

I am interested in the variance, which is defined as $$\text{Var}(X_t) = E[X_t^2] - \mu^2.$$ However, I read that $$\text{Var}(X_t) = \sigma_\epsilon^2/(1-\theta^2).$$

How is this derived?

$\endgroup$
  • 1
    $\begingroup$ The expression "$\text{Var}(X_t) = E[X_t^2] - \mu^2$" is not normally regarded as the definition of variance, but just equivalent to the usual definition, the second moment about the mean, $\text{Var}(X_t) = E[(X_t-\mu)^2]$. $\endgroup$ – Glen_b -Reinstate Monica Nov 13 '14 at 22:06
4
$\begingroup$

Compute the variance using the only information you have--the definition of the process itself--noting that adding a constant $c$ to any random variable does not change its variance:

$$\text{Var}(X_t) = \text{Var}(c + \theta X_{t-1} + \epsilon_t) = \text{Var}(\theta X_{t-1}) + \text{Var}(\epsilon_t) + 2\text{Cov}(\theta X_{t-1}, \epsilon_t).$$

Let's take the three terms at the end one at a time, from left to right. We can factor the $\theta$ out of $\text{Var}(\theta X_{t-1})$, where it must appear as $\theta^2$ (because variances are quadratic forms). I presume $\sigma_\epsilon^2$ is your name for $\text{Var}(\epsilon_t)$, so I will use it in the next expression. Now you probably were asked to assume that $\epsilon_t$ and $X_{t-1}$ are independent, or at least uncorrelated. In either of those cases the covariance term drops out because it is zero.

These considerations lead to the simplified formula

$$\text{Var}(X_t) = \theta^2\, \text{Var}(X_{t-1}) + \sigma_\epsilon^2 + 0 = \theta^2\, \text{Var}(X_{t-1}) + \sigma_\epsilon^2.$$

Finally, at some point you were probably invited to assume the process is second order stationary. That implies, inter alia, that

$$\text{Var}(X_t) = \text{Var}(X_{t-1}).$$

Plug this into the preceding equation to eliminate $\text{Var}(X_{t-1})$ and solve for $\text{Var}(X_t)$.

$\endgroup$
  • $\begingroup$ Thank you for those insights like second order stationary. Will I proceed in this same manner for AR(2) and MA process? $\endgroup$ – SKM Nov 13 '14 at 21:58
  • 1
    $\begingroup$ Certainly the same ideas should be borne in mind. Your next step might be to adapt them to computing the cross-covariance, $\text{Cov}(X_t, X_{t+h})$. The initial calculations are almost the same and the necessary assumptions are similar. Start with the case $h=1$ to get a sense of what's going on. $\endgroup$ – whuber Nov 13 '14 at 22:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.