In class my professor derived the variance for the AR(1) model in the case of $|\rho| < 1$. I am having some issues with derivation.

$$Var[X_k] = E[X_k^2] - E[X_k]^2 = E[X_k^2] - 0 = E[X_k^2]$$

$$E[X_k^2] = E[(\sum_{i=0}^\infty \rho^i \epsilon_{k-i})^2]$$

$$ = E[(\sum_{i=0}^\infty(\rho^i \epsilon_{k-i})^2 + 2\sum_{i<j}(\rho^i\epsilon_{k-i})(\rho^j \epsilon_{k-j})]$$

  1. I don't understand how my professor goes from the second line to the third line. Why can't we just write:

    $$E[(\sum_{i=0}^\infty \rho^i \epsilon_{k-i})^2] = E[\sum_{i=0}^\infty \rho^{2i} \epsilon_{k-i}^2]$$

  2. I also have absolutely no idea what is meant by $i < j$ in the second summation. Does it have something to do with the covariance matrix? Ie because when $i=j$ it's the same as the variance? I am basically totally lost on this step and why it is necessary.

The derivation then continues:

$$E[(\sum_{i=0}^\infty(\rho^i \epsilon_{k-i})^2 + 2\sum_{i<j}(\rho^i\epsilon_{k-i})(\rho^j \epsilon_{k-j})]$$

$$= \sum_{i=0}^\infty \rho^{2i} \sigma^2 + 2\sum_{i<j} \rho^{i+j} E[\epsilon_{k-i} \epsilon_{k-j}]$$

  1. I am confused by how we are able to pull $\rho$ out of the summation. I realize that it must be because $\rho$ is a constant, but I am confused by this because it depends on $i$ and $j$, so how can we treat it as we otherwise would a constant in this scenario?

I am comfortable with the rest of the derivation and the other steps I did not mention. Any help would be GREATLY appreciated. Thank you so much!

  • $\begingroup$ i<j simply means for a given value of j take the sum of the terms from i=0 to i=j-1. $\endgroup$ – Michael R. Chernick Mar 7 '18 at 8:30
  • $\begingroup$ For going from line 2 to line 3 you simply expand the terms in the square of a sum. As for the last part the expectation of the sum is the sum of the expectations (linearity property of expectations).. ($\rho^2$)$^i$ equals $\rho^2$$^i$ and for each index i E[($\epsilon_i$)$^2$] = $\sigma^2$. $\endgroup$ – Michael R. Chernick Mar 7 '18 at 8:45
  • $\begingroup$ So is the idea that the expectation of $\rho^{2i}$ is a constant, since $\rho$ is a constant? I guess that's where my confusion lies. $\endgroup$ – agra94 Mar 7 '18 at 21:24
  • $\begingroup$ $\rho$ is a parameter not a constant. For the purpose of deriving the result you can think of it as fixed but unknown. $\endgroup$ – Michael R. Chernick Mar 7 '18 at 21:40
  • $\begingroup$ Okay, so that is why we are able to essentially pull it out? It is fixed, the expectation of any "fixed" number will just be the number itself? like $E[\mu] = \mu$ because $\mu$, if we are referring to the normal distribution, is a parameter but it is fixed. Maybe that's a silly example but hopefully it gets my question across. $\endgroup$ – agra94 Mar 7 '18 at 21:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.