# How to recursively express an AR(p) process

We know that an AR($$1$$) process $$$$x_t=\rho x_{t-1}+u_t,\quad \lvert\rho\lvert<1$$$$ can be recursively expressed as $$\begin{eqnarray} x_t&=&\rho(\rho x_{t-2}+u_{t-1})+u_t\\ &\vdots&\\ x_t&=&\rho^t x_{0}+\sum\limits_{i=1}^t\rho^{t-i}u_i \end{eqnarray}$$ Can this be generalized? As in, is there a similar representation for an AR($$p$$) process?

• yes. see page 96 of this. stat.tamu.edu/~suhasini/teaching673/time_series.pdf. Note that, if you're into time series, this is a very under-rated time series "book" because it was never made into an actual text as far as I know. Mar 25, 2020 at 12:20

The truncated MA representation $$x_t = \rho^t x_{0}+\sum\limits_{i=0}^t\rho^{t-i}u_i$$ generalizes trivially to the AR$$(p)$$ case, with no restriction on the AR parameter. (The assumption $$\lvert\rho\lvert<1$$ is not necessary, for this representation.) Simply start with initial values $$x_0, x_{-1}, \cdots, x_{-p+1}$$, and iterate forward according to the model.

The causal MA$$(\infty)$$ representation $$x_t = \sum\limits_{i=0}^{\infty} \rho^{i}u_{t-i}$$ also generalizes to AR models where the AR polynomial has roots strictly outside the unit circle.

For an AR$$(p)$$ series $$\underbrace{ (1 - \rho_1 L - \rho_2 L^2 - \cdots \rho_p L^p) }_\text{\Phi(L)} x_t = u_t,$$ where the polynomial $$\Phi(z)$$ has roots strictly outside the unit circle, the $$\psi$$-weights in the causal MA$$(\infty)$$ representation $$x_t = \sum\limits_{i=0}^{\infty} \psi_{i}u_{t-i}$$ are the solutions to the difference equations

\begin{align*} \psi_0 &= 1 \\ \psi_1 - \rho_1 \psi_0 &= 0 \\ \psi_2 - \rho_1 \psi_1 - \rho_2 \psi_0 &= 0 \\ \vdots \\ \psi_{p-1} - \rho_1 \psi_{p-2} - \cdots \rho_{p-1} \psi_0 &= 0 \\ \psi_{t} - \rho_1 \psi_{t-1} - \rho_2 \psi_{t-2} \cdots \rho_{p} \psi_{t-p} &= 0, \;\; \forall t \geq p. \\ \end{align*}

The system can be solved like any linear homogeneous system of difference equations. The solution $$\{ \psi_i \}$$ is a linear combination of terms of the form $$r^{-t}$$ where $$r$$ is a root of the AR polynomial $$\Phi$$. (The cases of repeated or complex roots are ignored for simplicity. Same result holds.) The causality assumption ensures that $$x_t = \sum\limits_{i=0}^{\infty} \psi_{t-i}u_i$$ converges, as a random variable. (In the AR$$(1)$$ case, the causality condition is $$\lvert\rho\lvert<1$$.)

When the AR polynomial has roots possibly inside, but not on, the unit circle. The MA$$(\infty)$$ representation still exists but is not causal in general, i.e. it can be two-sided $$x_t = \sum\limits_{-\infty}^{\infty} \psi_{i}u_{t-i}.$$

For example, in the AR$$(1)$$ case with $$|\rho| > 1$$, then non-causal MA representation is given by iterating forward into the future $$x_t = \sum\limits_{- \infty < i \leq -1} (\frac{-1}{\rho })^{-i} u_{t-i}.$$

The existence of MA representations for AR (more generally, ARMA) series is not surprising. It's a special case of Wold representation, which says any weakly stationary time series has a two-sided MA representation.

• Thank you so much! This was highly useful.
– Carl
Mar 26, 2020 at 11:10