I have (or at least I think I have) a good intuition for why stationarity is a requirement for modeling $AR(p)$ models:

an $AR(p)$ model with coefficients $a_1,....,a_p$ :

$ Y_t = a_1Y_{t-1}+a_2Y_{t-2}+....a_qY_{t-q}$

such that one or more of the $|a_i| >1 $ , would diverge quickly.

But why is it a requirement for $MA(q)$ models? I don't have an intuitive grasp of why it is a requirement?

  • 1
    $\begingroup$ your question is unclear. you seem to start from two places: stationarity and the condition on the coefficient magnitude. what is your question about? also clarify what's "$a$" in your question $\endgroup$ – Aksakal Jan 18 '18 at 19:44
  • $\begingroup$ @Aksakal the stationarity condition is related to whether $|a| < 1 $ or not, so I'm not really starting from different places. I'll edit for clarity. $\endgroup$ – Skander H. Jan 18 '18 at 23:12

The autoregressive part indeed deals with stationarity. If it is statistically true that $|a| = 1$ there is evidence of a unit root and we get a whole bunch of unit-root associated problems. If $|a| > 1$, you are right, the series will quickly diverge.

For MA it indeed is a bit more tricky. We have to look at a property called invertibility: An invertible MA model is one that can be written as an infinite order AR model that converges so that the AR coefficients converge to 0 as we move infinitely back in time. We’ll demonstrate invertibility for the MA(1) model.

The MA(1) model can be written as $x_t - μ = w_t + θ_1w_{t-1}$.

If we let $z_t = x_t - μ$, then the MA(1) model is

\begin{equation} z_t=w_t+θ_1w_{t−1}. \end{equation} At time $t-1$, the model is $z_{t-1} = w_{t-1} + θ_1w_{t-2}$ which can be reshuffled to

\begin{equation} (2) w_{t−1}=z_{t−1}−θ_1w_{t−2}. \end{equation}

We then substitute relationship (2) for $w_{t-1}$ in equation (1)

\begin{equation} (3) z_t=w_t+θ_1(z_{t−1}−θ_1w_{t−2})=w_t+θ_1z_{t−1}−θ_2w_{t−2} \end{equation} At time t-2, equation (2) becomes

\begin{equation} (4) w_{t−2}=z_{t−2}−θ_1w_{t−3}. \end{equation} We then substitute relationship (4) for $w_{t-2}$ in equation (3)

\begin{equation} z_t=w_t+θ_1z_{t−1}−θ_{21}w_{t−2}=w_t+θ_1z_{t−1}−θ_{21}(z_{t−2}−θ_1w_{t−3})=w_t+θ_1z_{t−1}−θ_{21}z_{t−2}+θ_{31}w_{t−3} \end{equation} If we were to continue (infinitely), we would get the infinite order AR model

\begin{equation} z_t=w_t+θ_1z_{t−1}−θ_{21}z_{t−2}+θ_{31}z_{t−3}−θ_{41}z_{t−4}+… \end{equation} Note however, that if $|θ_1| ≥1$, the coefficients multiplying the lags of z will increase (infinitely) in size as we move back in time. To prevent this, we need |θ1| <1. This is the condition for an invertible MA(1) model.

We see that an AR(1) model can be converted to an infinite order MA model:

\begin{equation} x_t−μ=w_t+ϕ_1w_{t−1}+ϕ_{21}w_{t−2}+⋯+ϕ_{k1}w_{t−k}+⋯=\sum_{j = 0}^\infty ϕ_{j1}w_{t−j} \end{equation} This summation of past white noise terms is known as the causal representation of an AR(1). In other words, xt is a special type of MA with an infinite number of terms going back in time. This is called an infinite order MA or MA(∞). A finite order MA is an infinite order AR and any finite order AR is an infinite order MA.

And thus finally you are back at the stationary requirement for a AR model

  • 1
    $\begingroup$ please, use math markup $\endgroup$ – Aksakal Jan 18 '18 at 19:41
  • 1
    $\begingroup$ Yeahyeah, I quickly changed it to a more human-friendly format. Hope the answer helps! $\endgroup$ – Tom Jan 18 '18 at 19:52
  • $\begingroup$ What is $\theta_{21}$ etc.? $\endgroup$ – Jarle Tufto Jan 18 '18 at 20:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.