We know that an AR($1$) process \begin{equation} x_t=\rho x_{t-1}+u_t,\quad \lvert\rho\lvert<1 \end{equation} can be recursively expressed as \begin{eqnarray} x_t&=&\rho(\rho x_{t-2}+u_{t-1})+u_t\\ &\vdots&\\ x_t&=&\rho^t x_{0}+\sum\limits_{i=1}^t\rho^{t-i}u_i \end{eqnarray} Can this be generalized? As in, is there a similar representation for an AR($p$) process?

  • 1
    $\begingroup$ yes. see page 96 of this. stat.tamu.edu/~suhasini/teaching673/time_series.pdf. Note that, if you're into time series, this is a very under-rated time series "book" because it was never made into an actual text as far as I know. $\endgroup$
    – mlofton
    Mar 25, 2020 at 12:20

1 Answer 1


The truncated MA representation $$ x_t = \rho^t x_{0}+\sum\limits_{i=0}^t\rho^{t-i}u_i $$ generalizes trivially to the AR$(p)$ case, with no restriction on the AR parameter. (The assumption $\lvert\rho\lvert<1$ is not necessary, for this representation.) Simply start with initial values $x_0, x_{-1}, \cdots, x_{-p+1}$, and iterate forward according to the model.

The causal MA$(\infty)$ representation $$ x_t = \sum\limits_{i=0}^{\infty} \rho^{i}u_{t-i} $$ also generalizes to AR models where the AR polynomial has roots strictly outside the unit circle.

For an AR$(p)$ series $$ \underbrace{ (1 - \rho_1 L - \rho_2 L^2 - \cdots \rho_p L^p) }_\text{$\Phi(L)$} x_t = u_t, $$ where the polynomial $\Phi(z)$ has roots strictly outside the unit circle, the $\psi$-weights in the causal MA$(\infty)$ representation $x_t = \sum\limits_{i=0}^{\infty} \psi_{i}u_{t-i}$ are the solutions to the difference equations

\begin{align*} \psi_0 &= 1 \\ \psi_1 - \rho_1 \psi_0 &= 0 \\ \psi_2 - \rho_1 \psi_1 - \rho_2 \psi_0 &= 0 \\ \vdots \\ \psi_{p-1} - \rho_1 \psi_{p-2} - \cdots \rho_{p-1} \psi_0 &= 0 \\ \psi_{t} - \rho_1 \psi_{t-1} - \rho_2 \psi_{t-2} \cdots \rho_{p} \psi_{t-p} &= 0, \;\; \forall t \geq p. \\ \end{align*}

The system can be solved like any linear homogeneous system of difference equations. The solution $\{ \psi_i \}$ is a linear combination of terms of the form $r^{-t}$ where $r$ is a root of the AR polynomial $\Phi$. (The cases of repeated or complex roots are ignored for simplicity. Same result holds.) The causality assumption ensures that $x_t = \sum\limits_{i=0}^{\infty} \psi_{t-i}u_i$ converges, as a random variable. (In the AR$(1)$ case, the causality condition is $\lvert\rho\lvert<1$.)

When the AR polynomial has roots possibly inside, but not on, the unit circle. The MA$(\infty)$ representation still exists but is not causal in general, i.e. it can be two-sided $$ x_t = \sum\limits_{-\infty}^{\infty} \psi_{i}u_{t-i}. $$

For example, in the AR$(1)$ case with $|\rho| > 1$, then non-causal MA representation is given by iterating forward into the future $$ x_t = \sum\limits_{- \infty < i \leq -1} (\frac{-1}{\rho })^{-i} u_{t-i}. $$

The existence of MA representations for AR (more generally, ARMA) series is not surprising. It's a special case of Wold representation, which says any weakly stationary time series has a two-sided MA representation.

  • $\begingroup$ Thank you so much! This was highly useful. $\endgroup$
    – Carl
    Mar 26, 2020 at 11:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.