What is the difference between GARCH and ARMA?

I am confused. I don't understand the difference a ARMA and a GARCH process.. to me there are the same no ?

Here is the (G)ARCH(p, q) process

$$\sigma_t^2 = \underbrace{ \underbrace{ \alpha_0 + \sum_{i=1}^q \alpha_ir_{t-i}^2} _{ARCH} + \sum_{i=1}^p\beta_i\sigma_{t-i}^2} _{GARCH}$$

And here is the ARMA($p, q$):

$$X_t = c + \varepsilon_t + \sum_{i=1}^p \varphi_i X_{t-i} + \sum_{i=1}^q \theta_i \varepsilon_{t-i}.\,$$

Is the ARMA simply an extension of the GARCH, GARCH being used only for returns and with the assumption $r = \sigma\varepsilon$ where $\varepsilon$ follows a strong white process?

• In addition to fg nu's answer, the variance process in GARCH is time-varying. However, there is a trick here is that given a time-series of log-return of SP500, then to obtain the volatility process what should we do? Some people say that we need using the ARMA model to withdraw the residual series, then plug this residual series into the GARCH model to obtain the conditional variance process? Or directly plug the log-return plug the log-return process of SP500 into the GARCH model to obtain the conditional variance? – user26119 May 25 '13 at 16:09

You are conflating the features of a process with its representation. Consider the (return) process $(Y_t)_{t=0}^\infty$.

• An ARMA(p,q) model specifies the conditional mean of the process as

\begin{align} \mathbb{E}(Y_t \mid \mathcal{I}_t) &= \alpha_0 + \sum_{j=1}^p \alpha_j Y_{t-j}+ \sum_{k=1}^q \beta_k\epsilon_{t-k}\\ \end{align} Here, $\mathcal{I}_t$ is the information set at time $t$, which is the $\sigma$-algebra generated by the lagged values of the outcome process $(Y_t)$.

• The GARCH(r,s) model specifies the conditional variance of the process \begin{alignat}{2} & \mathbb{V}(Y_t \mid \mathcal{I}_t) &{}={}& \mathbb{V}(\epsilon_t \mid \mathcal{I}_t) \\ \equiv \,& \sigma^2_t&{}={}& \delta_0 + \sum_{l=1}^r \delta_j \sigma^2_{t-l} + \sum_{m=1}^s \gamma_k \epsilon^2_{t-m} \end{alignat}

Note in particular the first equivalence $\mathbb{V}(Y_t \mid \mathcal{I}_t)= \mathbb{V}(\epsilon_t \mid \mathcal{I}_t)$.

Aside: Based on this representation, you can write $$\epsilon_t \equiv \sigma_t Z_t$$ where $Z_t$ is a strong white noise process, but this follows from the way the process is defined.

• The two models (for the conditional mean and the variance) are perfectly compatible with each other, in that the mean of the process can be modeled as ARMA, and the variances as GARCH. This leads to the complete specification of an ARMA(p,q)-GARCH(r,s) model for the process as in the following representation \begin{align} Y_t &= \alpha_0 + \sum_{j=1}^p \alpha_j Y_{t-j} + \sum_{k=1}^q \beta_k\epsilon_{t-k} +\epsilon_t\\ \mathbb{E}(\epsilon_t\mid \mathcal{I}_t) &=0,\, \forall t \\ \mathbb{V}(\epsilon_t \mid \mathcal{I}_t) &= \delta_0 + \sum_{l=1}^r \delta_l \sigma^2_{t-l} + \sum_{m=1}^s \gamma_m \epsilon^2_{t-m}\, \forall t \end{align}
• Shouldn't you be conditioning on the information at time $t-1$ if all of the regressors are lagged? – Jase Dec 10 '12 at 6:32
• @Jase Note the definition, "Here, $\mathcal{I}_t$ is the information set at time $t$, which is the $\sigma$-algebra generated by the lagged values of the outcome process $(Y_t)$." That is, $\mathcal{I}_t = \sigma(Y_{t-1}, Y_{t-2}\ldots,)$. Some authors write this as $\mathcal{I}_{t-1}$ but that is counter to the notion of an information set at time $t$. – tchakravarty Dec 10 '12 at 7:04
• Nice! Do you know why we use the sigma-algebra and not a filtration? – Jase Dec 10 '12 at 7:17
• @Jase, the sequence of information sets $(\mathcal{I}_t)_{t=0}^\infty$ constitutes a filtration. – tchakravarty Dec 10 '12 at 7:20

Edit: I realized the answer was lacking and have thus provided a more precise answer (see below -- or maybe above). I have edited this one for factual mistakes and am leaving it for the record.

Different focus parameters:

• ARMA is a model for the realizations of a stochastic process imposing a specific structure of the conditional mean of the process.
• GARCH is a model for the realizations of a stochastic process imposing a specific structure of the conditional variance of the process.

Stochastic versus deterministic model:

• ARMA is a stochastic model in the sense that the dependent variable -- the realizations of the stochastic process -- is specified as a sum of a deterministic function of lagged dependent variable and lagged model error (the conditional mean) and a stochastic error term.
• GARCH is a deterministic model in the sense that the dependent variable -- the conditional variance of the process -- is a purely deterministic function of lagged variables.
• The conditional variance of GARCH process is a deterministic in your defined sense, buth the GARCH process is not, since $r_t=\sigma_t\varepsilon_t$, and $\varepsilon_t$ is independent of lags of $t$. – mpiktas Nov 20 '15 at 14:54
• @mpiktas, True. If the GARCH model contains two equations, one for conditional mean (an example of which you wrote above) and the other for conditional variance (which is intuitively, although not mathematically, "the main equation" of the model), my argument only applies to the latter equation. – Richard Hardy Nov 20 '15 at 15:19

ARMA

Consider $y_t$ that follows an ARMA($p,q$) process. Suppose for simplicity it has zero mean and constant variance. Conditionally on information $I_{t-1}$, $y_t$ can be partitioned into a known (predetermined) part $\mu_t$ (which is the conditional mean of $y_t$ given $I_{t-1}$) and a random part $u_t$:

\begin{aligned} y_t &= \mu_t + u_t; \\ \mu_t &= \varphi_1 y_{t-1} + \dotsc + \varphi_p y_{t-p} + \theta_1 u_{t-1} + \dotsc + \theta_q u_{t-q} \ \ \text{(known, predetermined)}; \\ u_t | I_{t-1} &~\sim D(0,\sigma^2) \ \ \text{(random)} \\ \end{aligned}

where $D$ is some density.

The conditional mean $\mu_t$ itself follows a process similar to ARMA($p,q$) but without the random contemporaneous error term: $$\mu_t = \varphi_1 \mu_{t-1} + \dotsc + \varphi_p \mu_{t-p} + (\varphi_1 + \theta_1) u_{t-1} + \dotsc + (\varphi_m + \theta_m) u_{t-m},$$ where $m:=\max(p,q)$; $\varphi_i=0$ for $i>p$; and $\theta_j=0$ for $j>q$. Note that this process has order ($p,m$) rather than ($p,q$) as does $y_t$.

We can also write the conditional distribution of $y_t$ in terms of its past conditional means (rather than past realized values) and model parameters as

\begin{aligned} y_t &\sim D(\mu_t,\sigma_t^2); \\ \mu_t &= \varphi_1 \mu_{t-1} + \dotsc + \varphi_p \mu_{t-p} + (\varphi_1 + \theta_1) u_{t-1} + \dotsc + (\varphi_m + \theta_m) u_{t-m}; \\ \sigma_t^2 &= \sigma^2, \end{aligned}

The latter representation makes the comparison of ARMA to GARCH and ARMA-GARCH easier.

GARCH

Consider $y_t$ that follows a GARCH($s,r$) process. Suppose for simplicity it has constant mean. Then

\begin{aligned} y_t &\sim D(\mu_t,\sigma_t^2); \\ \mu_t &= \mu; \\ \sigma_t^2 &= \omega + \alpha_1 u_{t-1}^2 + \dotsc + \alpha_s u_{t-s}^2 + \beta_1 \sigma_{t-1}^2 + \dotsc + \beta_r \sigma_{t-r}^2; \\ \frac{u_t}{\sigma_t} &\sim i.i.D(0,1), \\ \end{aligned}

where $u_t:=y_t-\mu_t$ and $D$ is some density.

The conditional variance $\sigma_t^2$ follows a process similar to ARMA($s,r$) but without the random contemporaneous error term.

ARMA-GARCH

Consider $y_t$ that has unconditional mean zero and follows an ARMA($p,q$)-GARCH($s,r$) process. Then

\begin{aligned} y_t &\sim D(\mu_t,\sigma_t^2); \\ \mu_t &= \varphi_1 \mu_{t-1} + \dotsc + \varphi_p \mu_{t-p} + (\varphi_1 + \theta_1) u_{t-1} + \dotsc + (\varphi_m + \theta_m) u_{t-m}; \\ \sigma_t^2 &= \omega + \alpha_1 u_{t-1}^2 + \dotsc + \alpha_s u_{t-s}^2 + \beta_1 \sigma_{t-1}^2 + \dotsc + \beta_r \sigma_{t-r}^2; \\ \frac{u_t}{\sigma_t} &\sim i.i.D(0,1), \\ \end{aligned}

where $u_t:=y_t-\mu_t$; $D$ is some density, e.g. Normal; $\varphi_i=0$ for $i>p$; and $\theta_j=0$ for $j>q$.

The conditional mean process due to ARMA has essentially the same shape as the conditional variance process due to GARCH, just the lag orders may differ (allowing for a nonzero unconditional mean of $y_t$ should not change this result significantly). Importantly, neither has random error terms once conditioned on $I_{t-1}$, thus both are predetermined.

The ARMA and GARCH processes are very similar in their presentation. The dividing line between the two is very thin since we get GARCH when an ARMA process is assumed for the error variance.