# Issues Manually Implementing ARMA GARCH

I have been working on manually implementing an ARMA GARCH (1,1) model but have been running into a few issues, namely a very large forecasted variance. I am hoping by outlining my process someone can catch a mistake.

First for ARMA, $$X_t = c + \phi X_{t-1} + \theta \epsilon_{t-1} + \epsilon_t$$ I assume some $$c, \phi$$, and $$\theta$$ and that $$\epsilon_0 = 0$$. Then by writing the above as $$\epsilon_t = X_t - c - \phi X_{t-1} -\theta\epsilon_{t-1}$$ I marched forward to construct a vector of $$\epsilon_i$$. The above process is optimized using OLS to return the set of $$c, \phi$$ and $$\theta$$, such that the vector of $$\epsilon_i$$ is minimized. I then use the optimized set of parameters to construct a forecast so that I can obtain a vector of ARMA residuals. Using my code, this is what an example output looks like: Do these residuals seem reasonable? Is my assumption that $$\epsilon_0 = 0$$ fair, or should it be excluded afterwards? Also, I currently have no constraints on my ARMA parameters, some sources claim that $$|\phi| \leq 1$$ should be imposed. Will this make a difference?

I then feed these residuals into a GARCH model, $$\sigma_{t}^2 = \omega + \alpha\epsilon_{t-1}^2 + \beta\sigma_{t-1}^2\\\epsilon_i = \sigma_i p_i$$ where $$p_i$$ is a process associated with some (not necessarily Gaussian) distribution with mean 0 and scale 1.

To calculate the GARCH parameters I use MLE. I assume $$\sigma^2_0 = 0$$. For each time $$n \in [0, t]$$ I march forward using the GARCH equation and the ARMA residuals until I reach $$\sigma_n^2$$. I then evaluate $$\frac{\epsilon_n}{\sigma_n}$$ at the distribution's log PDF to obtain a probability. The sum of these values from $$T=1$$ to $$T = t$$ is my overall MLE. I also assume some parameters for the distribution (if there are any beyond the mean and scale). I then repeat this process so that I optimize both my distribution parameters and also my GARCH parameters. I require that all GARCH parameters be non-negative and that $$\alpha + \beta = 1$$

As noted, this process leads to incredibly large values for the forecasted variances (around 4000-5000). I notice it also chooses a rather large value for $$\omega$$ (somewhere around 8-30), and almost always chooses $$\alpha = 0, \beta = 1$$. For what it is worth, I am using the NLOPT package for the optimization and the COBYLA algorithm.

Does anyone see where I might have made a mistake or if I have made a false assumption? Thanks for reading!

• (Perhaps) a minor quibble: distribution with mean 0 and scale 1: scale 1 should be variance 1, as scale is not always equal to variance. Another one which I already mentioned under another question of yours: $\sigma_0^2$ is an unusual assumption, especially given it is impossible. The common thing to do is to set $\sigma_0^2=\hat{\sigma}^2$. I would also temporarily drop the ARMA part and experiment with pure GARCH to see what is wrong with it. Apr 29, 2021 at 18:21
• @RichardHardy Thank you. I mentioned scale because some distributions I am working with (i.e. NIG) have a scale parameter. In my code I have set the scale equal to 1, perhaps this could be a source of error? In regards to $\hat{\sigma}^2$, how is that calculated? Apr 29, 2021 at 18:24
• Indeed, be careful with equating scale with variance. I am not sure about NIG, but I would surely double check if I were you. (It would be funny and cool if that solved the problem!) $\hat\sigma^2$ a naive estimate of the unconditional variance, i.e. the sample variance. Apr 29, 2021 at 18:26
• @RichardHardy Thank you. I will make these changes and hopefully that solves the issue. In regards to the ARMA portion, does everything look reasonable? Apr 29, 2021 at 18:50
• @RichardHardy Update: I seem to be getting more reasonable forecasts due to the PDF change, but my GARCH terms seem more or less the same even with the updated $\sigma_0^2$. Apr 29, 2021 at 18:53

Also, $$\sigma_0^2=0$$ is an unusual assumption, especially given that zero variance is impossible. The common thing to do is to set $$\sigma_0^2:=\hat\sigma^2$$ where $$\hat\sigma^2$$ is the sample variance.
• It turns out both the mean and variance of the NIG distribution are functions of other parameters, so I have changed that in my code. I have also set $\sigma_0^2:=\hat\sigma^2$ as my initial variance. However not much has changed in that I am still getting unreasonable forecast values and rather large variances. I compared my ARMA results with an existing package and visually there is not that much of a difference, so I assume the problem is in either GARCH or the distribution optimization, but I cannot tell which. Apr 29, 2021 at 19:57
• @CBBAM I suggest that you first try to fit a pure GARCH-model and then extend your results to the ARMA-GARCH case. Also, why do you set the constrain $\alpha+\beta=1$ ? In this case you have an IGARCH(1,1)-model, i.e. the unconditional variance does not exist.