I'm looking for a proof for the stationarity of an AR(p) process, I know a stationary process $Yt$ must fulfill the following conditions:

$(1)$-$E(Y_t)=m$ for all values of $t$.

$(2)$-$Var(Y_t)=\sigma^2$ for all values of $t$.

$(3)$-$cov(Y_t,Y_s)=\gamma(|t-s|)$ for every $t$ and $s$.

First I'll define an AR(p) process $Y_t$ as follows:

$\phi(B)Y_t=\epsilon_t$ with: $BY_t=Y_{t-1}$, $\epsilon_t$ a white noise and $\phi(B)=1-\alpha_1B-...-\alpha_pB^p$.

The condition that I read about in several posts is:

$(4)$ If the modulus of each root of $\phi$ is strictly greater than 1 then the AR(p) process as defined before is stationary.

So, what I'm looking for is a proof for: $(4)$$<=>$$(1)$&$(2)$&$(3)$

  • $\begingroup$ Are you sure about strictly greater part? if I recall correctly, as long as the roots are outside the unit disk, the solution is stationary $\endgroup$ – V. Aslanyan Apr 12 '19 at 17:40

We first start with AR(1) with $|\phi|<1$. We write $$x_t=\phi^{-1}x_{t-1}+w_{t}=\phi(\phi x_{t-2}+w_{t-1})+w_{t}=\dots=\phi^{k}x_{t-k}+\sum_{j=1}^{k-1}\phi^{j}w_{t-j}$$ We assume $sum_t var(x_t)< \infty$, and we can write $$x_t=\sum \phi^j w_{t-j}$$ now$$E(x_t)=\sum \phi^j E(w_{t-j})=0 $$ autocovariance function is $$cov(x_{t+h},x_t)=E((\sum_{j}\phi^j w_{t+h-j})(\sum_k\phi^k w_{t-k}) )=E((w_{t+h}+\dots+\phi^h w_t+\phi^{h+1}w_{t-1}+\dots))\\(w_t+\phi w_{t-1}+\dots)=\sigma^2_w\sum_j \phi^{h+j}\phi^j=\sigma^2_w\phi^h\sum_j\phi^{2j}=\frac{\sigma_w^2\phi^k}{1-\phi^2} $$

The other way: $$x_t=\phi x_{t-1}+w_t\rightarrow(1-\phi B)x_t=w_t\rightarrow1-\phi B=0\rightarrow x=\frac{1}{\phi}\rightarrow |\frac{1}{\phi}|>1\rightarrow|\phi|<1$$ When $|\phi|>1$, the process is called explosive and one can prove that every explosive process has a stochastically equivalent causal process.

To generalize for AR(p):

$$x_t=\phi_1x_{t-1}+\phi_2x_{t-2}+\dots+\phi_px_{t-p}+w_t$$ We rewrite this as an AR(1) model $y_t=Fy_{t-1}+w_t$

[ \begin{bmatrix} x_{t} \\ x_{t-1} \\ \dots\\ x_{t-h+1} \end{bmatrix}=\begin{bmatrix} \phi_{1} & \phi_{2} & \phi_{3} & \dots & \phi_{p} \\ 1 & 0 & 0 & \dots & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & 0 & \dots & 1 \end{bmatrix}\begin{bmatrix} x_{t-1} \\ x_{t-2} \\ \dots \\ x_{t-p} \end{bmatrix}+\begin{bmatrix} w_{t} \\ 0 \\ \dots \\ 0 \end{bmatrix} ]

Lets call thus augmented AR(1) notation. We can write the AR model recursively $$y_t=F^ty_0+w_t+Fw_{t-1}+F^2w_{t-2}+\dots+F^{t-1}w_1+F^tw_0$$. Recall from eigenvalue decomposition: $F=Q\Lambda Q^{-1}$ and $F^j=Q\Lambda^jQ^{-1}$ where $\Lambda^j$ is \begin{bmatrix} \lambda_{1}^j & 0 & 0 & \dots & 0 \\ 0 & \lambda_{2}^j & 0 & \dots & 0 \\ \dots \\ 0 & 0 & 0 & \dots & \lambda_p^j \end{bmatrix} and is stable when $|\lambda_i|<1$ for all i.

The actual eigenvalues of F satisfy $$\lambda^p-\phi_1\lambda^{p-1}-\phi_2\lambda^{p-2}-\dots-\phi_{p-1}\lambda-\phi_p=0$$ So the eigenvalues are the reciprocal of the values that solve the characteristic polynomial of the AR(p), so the roots should be outside the unit circle

  • $\begingroup$ Thanks for answering, I understand what you did for the AR(1)...Although could you please further detail the 'augmented AR(1)' as you called it. First of all shouldn't the last element of $y_t$ be $x_{t−p}$ and why would the eigenvalues of F being lower than 1 in modulus an indication that the process is stationary? Also consider changing in the recursive formula from $y+t$ to $y_t$ for future viewers $\endgroup$ – Youssef Esseddiq Apr 12 '19 at 23:30
  • $\begingroup$ yes, you are correct. Typing in Latex syntax is not my forte. Eigenvalues of Fs being less than 1 implies that F is stable which implies that y satisfies causality assumption. Since it satisfies causality, y_t is stationary $\endgroup$ – V. Aslanyan Apr 12 '19 at 23:42
  • $\begingroup$ I unwrapped all the elements for y to be in terms of $y_0$ and $w_j$s $\endgroup$ – V. Aslanyan Apr 12 '19 at 23:43
  • $\begingroup$ Could you lead me to good sources where I can read about causality and causality assumption? $\endgroup$ – Youssef Esseddiq Apr 13 '19 at 0:00
  • $\begingroup$ Chapter 3of this book covers pretty much everything ARIMA related on undergraduate level $\endgroup$ – V. Aslanyan Apr 15 '19 at 15:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.