Difference between MA and AR

Question

I fail to see the difference between

Moving Average (MA): $x_t=\epsilon_t+β_1\epsilon_{t−1}+…+β_q\epsilon_{t−q}$

Autoregressive (AR):   $x_t=\epsilon_t+β_1x_{t−1}+…+β_qx_{t−q}$

$x_t$ is the observed value and $\epsilon$ is the residual. But since both assume a stationary time series with $\mu = 0$, isn't the residual and the observed value identical? I.e., $\epsilon_i = x_i - \mu = x_i$?

This would imply that MA and AR are identical, but I can see everyone using these expressions to explain that they differ, so what am I missing here?

Answer 1

A key difference which I failed to appreciate: the MA model predictions of $x_t$ include $\epsilon_{t-1}$ in its computation whereas the AR model only predicts based on $x_{t-1}$ without (explicit) regard as to whether the prediction at $t-1$ was under- or over-estimating $x_{t-1}$. Fleshing out what @user1587692 highlighted, the MA model averages across innovations ($\epsilon$, i.e., the "novelty" that the MA model failed to catch, even with its lag(1) component). The AR model, on the other hand, average previous observations (residuals, i.e., $x$ when $\mu = 0$) without splitting the time-series part and innovation part.

To make this super clear, I made a small dataset with $\mu = 0$, $\beta_1 = 0.5$ and $\beta_2 = 0.2$ (latter only used for MA(2) and AR(2)). Here, $x$ is called "observed" and $\epsilon$ is called "MA(1) error". The yellow cells are "helpers", typed in to make the predictions work for observation $x_1$.

To me, this also gives a better intuition why:

1. The covariance between $x_t$ back until $x_{t-order}$ is positive and zero further back. This is due to the i.i.d. residuals (innovations on top of the non-i.i.d. autocorrelation) that contribute within the averaging window but has, by definition of being white noise, no information about $x_t$ further back.

2. AR models have (positive or negative) autocorrelations further back. This is just the time-series autocorrelation (dampened by the i.i.d. innovations on top).

3. Why AR and MA can be viewed as re-parameterizations of each other. No information is added or removed in either model. However, you cannot transform one to the other at any particular time ($t$, column in the table above) - you need to look further back to make an estimate. That's why you need a high order AR to estimate a low-order MA. It takes a lot of $x$ to estimate $\epsilon$ if you can't model it directly.

4. Similarly, it takes a high order MA to estimate a low-order AR if you don't model $x$ directly (condition it out).

Answer 2

You should consider the $\epsilon_t$ innovations rather than residuals. In the MA case, you average across the recent innovations, whereas in the AR case you average across the recent observations. Even if the models are stationary and have no deterministic terms, the innovations and the observations are different. For example, suppose the $\epsilon_t$ are i.i.d.. Then, in the MA(1) case, the covariance between $x_t$ and $x_{t-1}$ is still positive! However, the covariance between $x_t$ and $x_{t-2}$ will still be zero. In the AR(1) case on the other hand, all autocorrelations are nonzero but decaying exponentially.

Answer 3

A finite AR model can be expressed as an MA model and vice-versa , If one has an ar(1) model with coefficient .333333333 then the models are (nearly) identical .

Consider the case for an ar(1) with coefficient of .3

$x_t=ϵ_t+.3*x_{t−1}$

since $x_{t-1}=e_{t-1}+.3*x_{t-2}$ we can substitute for $x_{t-1}$ and get

$x_t=ϵ_t+.3*[e_{t-1} + .3*x_{t-2}]$

$\implies x_t=ϵ_t+.3*e_{t-1} + .09*x_{t-2}$

and

$x_t = ϵ_t+.3*e_{t-1} + .09*e_{t-2} + .027*x_{t-3}]$

etc .

The "reason" for selecting an ar model versus an ma model is simply for parsimony.

They are identical when q=1 and B1=.333333333 and for other cases for q>1

Hope this helps ....