# Conceptual questions: Variance of a process

Wikepedia, at Variance of Autoregressive model, mentions an expression of variance for an AR(1) process. I am learning signal processing (beginner level) and facing difficulty in understanding some basic relations. It shall be really helpful if the following doubts are answered.

An AR(1) process is defined by $$X_t = c+\theta X_{t-1} + \epsilon_t$$

where $\epsilon_t$ is a zero mean white Gaussian noise. We may compute expected values

$$E[X_t] = E[c] + \theta E[X_{t-1}]+E[\epsilon_t]$$

implying $E[X_t] = \mu = c/(1-\theta)$.

I am interested in the variance, which is defined as $$\text{Var}(X_t) = E[X_t^2] - \mu^2.$$ However, I read that $$\text{Var}(X_t) = \sigma_\epsilon^2/(1-\theta^2).$$

How is this derived?

• The expression "$\text{Var}(X_t) = E[X_t^2] - \mu^2$" is not normally regarded as the definition of variance, but just equivalent to the usual definition, the second moment about the mean, $\text{Var}(X_t) = E[(X_t-\mu)^2]$. – Glen_b Nov 13 '14 at 22:06

Compute the variance using the only information you have--the definition of the process itself--noting that adding a constant $$c$$ to any random variable does not change its variance:

$$\text{Var}(X_t) = \text{Var}(c + \theta X_{t-1} + \epsilon_t) = \text{Var}(\theta X_{t-1}) + \text{Var}(\epsilon_t) + 2\text{Cov}(\theta X_{t-1}, \epsilon_t).$$

Let's take the three terms at the end one at a time, from left to right. We can factor the $$\theta$$ out of $$\text{Var}(\theta X_{t-1})$$, where it must appear as $$\theta^2$$ (because variances are quadratic forms). I presume $$\sigma_\epsilon^2$$ is your name for $$\text{Var}(\epsilon_t)$$, so I will use it in the next expression. Now you probably were asked to assume that $$\epsilon_t$$ and $$X_{t-1}$$ are independent, or at least uncorrelated. In either of those cases the covariance term drops out because it is zero.

These considerations lead to the simplified formula

$$\text{Var}(X_t) = \theta^2\, \text{Var}(X_{t-1}) + \sigma_\epsilon^2 + 0 = \theta^2\, \text{Var}(X_{t-1}) + \sigma_\epsilon^2.$$

Finally, at some point you were probably invited to assume the process is second order stationary. That implies, inter alia, that

$$\text{Var}(X_t) = \text{Var}(X_{t-1}).$$

Plug this into the preceding equation to eliminate $$\text{Var}(X_{t-1})$$ and solve for $$\text{Var}(X_t)$$.

• Thank you for those insights like second order stationary. Will I proceed in this same manner for AR(2) and MA process? – SKM Nov 13 '14 at 21:58
• Certainly the same ideas should be borne in mind. Your next step might be to adapt them to computing the cross-covariance, $\text{Cov}(X_t, X_{t+h})$. The initial calculations are almost the same and the necessary assumptions are similar. Start with the case $h=1$ to get a sense of what's going on. – whuber Nov 13 '14 at 22:13