Derivation of the ARMA model as acombination of the AR and MA models

Question

On the Wikipedia article on the ARMA model, its derivation is simplified as a combination of the AR and MA models:

AR

$$ X_t = c + \sum_{i=1}^p \varphi_i X_{t-i} + \varepsilon_t $$

MA

$$ X_t = \mu + \varepsilon_t + \sum_{i=1}^q \theta_i \varepsilon_{t-i} $$

ARMA

$$ X_t = c + \varepsilon_t + \sum_{i=1}^p \varphi_i X_{t-i} + \sum_{i=1}^q \theta_i \varepsilon_{t-i} $$

Sum of AR and MA

$$ 2X_t = \mu + c + 2\varepsilon_t + \sum_{i=1}^p \varphi_i X_{t-i} + \sum_{i=1}^q \theta_i \varepsilon_{t-i} $$

At first glance it seems almost like they are simply summed together to form this model, however, $\mu$ is gone, and $\varepsilon_t$ and $X_t$ should be doubled. Clearly this is not an entirely correct interpretation, but it's still close.

Is there a way to explain this discrepancy between the sum of AR and MA and the ARMA model, or is there another more natural way of deriving this model?

Answer 1

It's much easier to do this "derivation" with lag operator: $$Lz_t=z_{t-1}$$ and $$\phi(L)=1+\sum_{i=1}^q\phi_iL^i$$ $$\theta(L)=1-\sum_{i=1}^q\theta_iL^i$$

This way MA(q) is $$x_t=\phi(L)\varepsilon_t=c+\varepsilon_t+\sum_{i=1}^q\phi_i\varepsilon_{t-i}$$ and AR(p) is $$\theta(L)x_t=x_t-\sum_{i=1}^p\theta_ix_{t-i}=c$$. You can combine them as you wish. For instance, ARMA(p,q) is $$\theta(L)x_t=c+\phi(L)\varepsilon_t$$

Let's see how ARMA(1,2) works out: $$x_t-\theta_1x_{t-1}=c+\varepsilon_t+\phi_1\varepsilon_{t-1}+\phi_2\varepsilon_{t-2}$$ $$x_t=c+\theta_1x_{t-1}+\varepsilon_t+\phi_1\varepsilon_{t-1}+\phi_2\varepsilon_{t-2}$$

Note, that in MA process, the mean is equal to a constant: $$E[x_t]=c$$ In AR or ARMA this is not true: $$E[x_t]\ne c$$ That is why sometimes the connstant in MA process is denoted with $\mu$ to allude to the common symbol for mean in statistics.

Answer 2

No, there is no way to explain this discrepancy because it does not make sense to sum together an AR and MA model in the way you have, because $X_t$ cannot be both an AR model and an MA model at the same time. If you write down $$ X_t = \mu + \varepsilon_t + \sum_{i=1}^q \theta_i \varepsilon_{t-i} $$ with $q$ finite, then you cannot also write down $$ X_t = c + \sum_{i=1}^p \varphi_i X_{t-i} + \varepsilon_t, $$ with $p$ finite, because it is a contradiction. You can call one of these processes $Y_t$, and then write the sum as $W_t$, however.

...or is there another more natural way of deriving this model?

Yes, both of these models are types of linear processes.