How to recursively express an AR(p) process We know that an AR($1$) process 
\begin{equation}
x_t=\rho x_{t-1}+u_t,\quad \lvert\rho\lvert<1
\end{equation}
can be recursively expressed as
\begin{eqnarray}
x_t&=&\rho(\rho x_{t-2}+u_{t-1})+u_t\\
&\vdots&\\
x_t&=&\rho^t x_{0}+\sum\limits_{i=1}^t\rho^{t-i}u_i
\end{eqnarray}
Can this be generalized? As in, is there a similar representation for an AR($p$) process?
 A: The truncated MA representation
$$
x_t = \rho^t x_{0}+\sum\limits_{i=0}^t\rho^{t-i}u_i
$$
generalizes trivially to the AR$(p)$ case, with no restriction on the AR parameter. (The assumption $\lvert\rho\lvert<1$ is not necessary, for this representation.) Simply start with initial values $x_0, x_{-1}, \cdots, x_{-p+1}$, and iterate forward according to the model.
The causal MA$(\infty)$ representation
$$
x_t = \sum\limits_{i=0}^{\infty} \rho^{i}u_{t-i}
$$
also generalizes to AR models where the AR polynomial has roots strictly outside the unit circle.
For an AR$(p)$ series
$$
\underbrace{ (1 - \rho_1 L - \rho_2 L^2 - \cdots \rho_p L^p) }_\text{$\Phi(L)$} x_t = u_t,
$$ where the polynomial $\Phi(z)$ has roots strictly outside the unit circle, the $\psi$-weights in the causal MA$(\infty)$ representation $x_t = \sum\limits_{i=0}^{\infty} \psi_{i}u_{t-i}$ are the solutions to the difference equations
\begin{align*}
\psi_0 &= 1 \\
\psi_1 - \rho_1 \psi_0 &= 0 \\
\psi_2 - \rho_1 \psi_1 - \rho_2 \psi_0 &= 0 \\
\vdots \\
\psi_{p-1} - \rho_1 \psi_{p-2} - \cdots \rho_{p-1} \psi_0 &= 0 \\
\psi_{t} - \rho_1 \psi_{t-1} - \rho_2 \psi_{t-2} \cdots \rho_{p} \psi_{t-p} &= 0, \;\; \forall t \geq p. \\
\end{align*}
The system can be solved like any linear homogeneous system of difference equations. The solution $\{ \psi_i \}$ is a linear combination of terms of the form $r^{-t}$ where $r$ is a root of the AR polynomial $\Phi$. (The cases  of repeated or complex roots are ignored for simplicity. Same result holds.) The causality assumption ensures that
$x_t = \sum\limits_{i=0}^{\infty} \psi_{t-i}u_i$ converges, as a random variable. (In the AR$(1)$ case, the causality condition is $\lvert\rho\lvert<1$.)
When the AR polynomial has roots possibly inside, but not on, the unit circle. The MA$(\infty)$ representation still exists but is not causal in general, i.e. it can be two-sided
$$
x_t = \sum\limits_{-\infty}^{\infty} \psi_{i}u_{t-i}.
$$
For example, in the AR$(1)$ case with $|\rho| > 1$, then non-causal MA representation is given by iterating forward into the future
$$
x_t = \sum\limits_{- \infty < i \leq -1} (\frac{-1}{\rho })^{-i} u_{t-i}.
$$
The existence of MA representations for AR (more generally, ARMA) series is not surprising. It's a special case of Wold representation, which says any weakly stationary time series has a two-sided MA representation.
