# Derivation of the ARMA model as acombination of the AR and MA models

On the Wikipedia article on the ARMA model, its derivation is simplified as a combination of the AR and MA models:

### AR

$$X_t = c + \sum_{i=1}^p \varphi_i X_{t-i} + \varepsilon_t$$

### MA

$$X_t = \mu + \varepsilon_t + \sum_{i=1}^q \theta_i \varepsilon_{t-i}$$

### ARMA

$$X_t = c + \varepsilon_t + \sum_{i=1}^p \varphi_i X_{t-i} + \sum_{i=1}^q \theta_i \varepsilon_{t-i}$$

## Sum of AR and MA

$$2X_t = \mu + c + 2\varepsilon_t + \sum_{i=1}^p \varphi_i X_{t-i} + \sum_{i=1}^q \theta_i \varepsilon_{t-i}$$

At first glance it seems almost like they are simply summed together to form this model, however, $\mu$ is gone, and $\varepsilon_t$ and $X_t$ should be doubled. Clearly this is not an entirely correct interpretation, but it's still close.

Is there a way to explain this discrepancy between the sum of AR and MA and the ARMA model, or is there another more natural way of deriving this model?

It's much easier to do this "derivation" with lag operator: $$Lz_t=z_{t-1}$$ and $$\phi(L)=1+\sum_{i=1}^q\phi_iL^i$$ $$\theta(L)=1-\sum_{i=1}^q\theta_iL^i$$

This way MA(q) is $$x_t=\phi(L)\varepsilon_t=c+\varepsilon_t+\sum_{i=1}^q\phi_i\varepsilon_{t-i}$$ and AR(p) is $$\theta(L)x_t=x_t-\sum_{i=1}^p\theta_ix_{t-i}=c$$. You can combine them as you wish. For instance, ARMA(p,q) is $$\theta(L)x_t=c+\phi(L)\varepsilon_t$$

Let's see how ARMA(1,2) works out: $$x_t-\theta_1x_{t-1}=c+\varepsilon_t+\phi_1\varepsilon_{t-1}+\phi_2\varepsilon_{t-2}$$ $$x_t=c+\theta_1x_{t-1}+\varepsilon_t+\phi_1\varepsilon_{t-1}+\phi_2\varepsilon_{t-2}$$

Note, that in MA process, the mean is equal to a constant: $$E[x_t]=c$$ In AR or ARMA this is not true: $$E[x_t]\ne c$$ That is why sometimes the connstant in MA process is denoted with $\mu$ to allude to the common symbol for mean in statistics.

• Where is the $\mu$ from the AR model? Aug 21 '18 at 16:23
• Forget about $\mu$, it's just constant $c$, which you can call whatever you want. You do not literally add AR to MA, like you were trying to do. You add AR or MA components like I show into the context of the ARIMA equation. The reason in MA you call a constant $\mu$ is to allude to the mean of the series, while in AR the constant $c$ is not equal to the mean usually. Aug 21 '18 at 16:33
• So to be clear, the constant $c$ in the ARMA model is different from the constant in the AR model? Aug 21 '18 at 17:30
• The meaning of the constant depends on the model specification indeed. Actually, in ARMA the meaning is closer to AR than to MA. Aug 21 '18 at 17:57

No, there is no way to explain this discrepancy because it does not make sense to sum together an AR and MA model in the way you have, because $X_t$ cannot be both an AR model and an MA model at the same time. If you write down $$X_t = \mu + \varepsilon_t + \sum_{i=1}^q \theta_i \varepsilon_{t-i}$$ with $q$ finite, then you cannot also write down $$X_t = c + \sum_{i=1}^p \varphi_i X_{t-i} + \varepsilon_t,$$ with $p$ finite, because it is a contradiction. You can call one of these processes $Y_t$, and then write the sum as $W_t$, however.

...or is there another more natural way of deriving this model?

Yes, both of these models are types of linear processes.