Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I have (or at least I think I have) a good intuition for why stationarity is a requirement for modeling $AR(p)$ models:
an $AR(p)$ model with coefficients $a_1,....,a_p$ :
$ Y_t = a_1Y_{t-1}+a_2Y_{t-2}+....a_qY_{t-q}$
such that one or more of the $|a_i| >1 $ , would diverge quickly.
But why is it a requirement for $MA(q)$ models? I don't have an intuitive grasp of why it is a requirement?
The autoregressive part indeed deals with stationarity. If it is statistically true that $|a| = 1$ there is evidence of a unit root and we get a whole bunch of unit-root associated problems. If $|a| > 1$, you are right, the series will quickly diverge.
For MA it indeed is a bit more tricky. We have to look at a property called invertibility: An invertible MA model is one that can be written as an infinite order AR model that converges so that the AR coefficients converge to 0 as we move infinitely back in time. We’ll demonstrate invertibility for the MA(1) model.
The MA(1) model can be written as $x_t - μ = w_t + θ_1w_{t-1}$.
If we let $z_t = x_t - μ$, then the MA(1) model is
\begin{equation} z_t=w_t+θ_1w_{t−1}. \end{equation} At time $t-1$, the model is $z_{t-1} = w_{t-1} + θ_1w_{t-2}$ which can be reshuffled to
\begin{equation} (2) w_{t−1}=z_{t−1}−θ_1w_{t−2}. \end{equation}
We then substitute relationship (2) for $w_{t-1}$ in equation (1)
\begin{equation} (3) z_t=w_t+θ_1(z_{t−1}−θ_1w_{t−2})=w_t+θ_1z_{t−1}−θ_2w_{t−2} \end{equation} At time t-2, equation (2) becomes
\begin{equation} (4) w_{t−2}=z_{t−2}−θ_1w_{t−3}. \end{equation} We then substitute relationship (4) for $w_{t-2}$ in equation (3)
\begin{equation} z_t=w_t+θ_1z_{t−1}−θ_{21}w_{t−2}=w_t+θ_1z_{t−1}−θ_{21}(z_{t−2}−θ_1w_{t−3})=w_t+θ_1z_{t−1}−θ_{21}z_{t−2}+θ_{31}w_{t−3} \end{equation} If we were to continue (infinitely), we would get the infinite order AR model
\begin{equation} z_t=w_t+θ_1z_{t−1}−θ_{21}z_{t−2}+θ_{31}z_{t−3}−θ_{41}z_{t−4}+… \end{equation} Note however, that if $|θ_1| ≥1$, the coefficients multiplying the lags of z will increase (infinitely) in size as we move back in time. To prevent this, we need |θ1| <1. This is the condition for an invertible MA(1) model.
We see that an AR(1) model can be converted to an infinite order MA model:
\begin{equation} x_t−μ=w_t+ϕ_1w_{t−1}+ϕ_{21}w_{t−2}+⋯+ϕ_{k1}w_{t−k}+⋯=\sum_{j = 0}^\infty ϕ_{j1}w_{t−j} \end{equation} This summation of past white noise terms is known as the causal representation of an AR(1). In other words, xt is a special type of MA with an infinite number of terms going back in time. This is called an infinite order MA or MA(∞). A finite order MA is an infinite order AR and any finite order AR is an infinite order MA.
And thus finally you are back at the stationary requirement for a AR model
