# Specifying an ARMA-GARCH model without rugarch

There have been probably hundreds of these posts, but I have yet to see any sort of answer to how to actually go about this. I need to fit a ARMA-GARCH model to my data because I cannot assume my data have a constant mean.

I cannot use rugarch as most of the posts addressing this have used so I have to be able to do this myself using Python and PyFlux.

Here are the posts I have looked at:

My initial assumption was that I could fit an ARIMA model to the data and then fit a GARCH model to the residuals. This seems to be a common thing people do, but according to this post by @RichardHardy it is incorrect and inconsistent - though I can't find any evidence other than his posts indicating this. Unfortunately this has left me with more questions than answers because I can't just fall back to rugarch to solve this. I also haven't been able to find any literature on the process of fitting an ARMA-GARCH model that doesn't rely on handwaving rugarch as the solution. I'm hoping (perhaps Richard Hardy or someone who understands this stuff) could answer these questions:

1. What is meant by "fitting simultaneously"? To me this feels like a chicken and egg problem.

2. Would it be possible to fit the ARIMA model and then a GARCH model on the residuals if you had an adjusted Lijung-Box test and ACF/PACFs?

3. Once the model is fitting, how is a forecast made from the the two models?

• Did you try to write down the likelihood of the model you define? If you know how to write down the model likelihood, then you can apply some techniques, for example, MCMC to estimate the parameters, this is what do you mean by 'simultaneously'. Also, you can use two steps methods as you mentioned in your quiz 2. Finally, once you get the parameters for the whole model, then you can estimate the mean at $t$ and the variance as well, that's prediction. Dec 20, 2016 at 1:58
• @Fly_back I have never done that before. To be honest I mostly have dealt with R and have been self teaching time series analysis. Would it literally be the sum of an ARIMA(p,i,q) and GARCH(P,Q) model together, and then I use MLE or something to fit the model?
– user124589
Dec 20, 2016 at 3:11
• In this case, I suggest you start from MATLAB, it has a very good introduction on ARIMA model fitting, very friendly. I am not good at MLE method and I can not give more advices. Most of my experience are estimate parameters by MCMC. Dec 20, 2016 at 4:24

You touch upon two main issues: estimation and model selection.

Estimation

For a given model specification, you may

1. Either write down the likelihood function and feed it into a generic optimizer (such as the function optim in R);
2. Or use an existing function that takes the model specification (e.g. ARMA(p,q)-GARCH(s,r)), "writes the likelihood" for you and optimizes it (such as the function ugarchfit in the "rugarch" package in R).

Both ways are fine:

1. The first one allows you to dig deep into model fitting and perhaps alter model specification beyond what is available in the standard functions; however, it is time consuming and the way you are fitting might not be optimized for speed (while the standard functions typically are), unless you are a knowledgeable programmer with experience in optimization;
2. The second one allows for a convenient and fast estimation without the need to understand how exactly it is done (the latter can also be a drawback depending on what you wish to achieve).

The likelihood functions of ARMA and GARCH are available in time series textbooks such as Hamilton "Time Series Analysis" (2004) or Tsay "Analysis of Financial Time Series" (2005), among other. Hopefully, the likelihood of ARMA-GARCH is also available somewhere, but I do not have a refence handy. You could try textbooks, lecture notes or maybe software documentation (R, Stata, Matlab). (Please post any references here in the comments, I will appreciate it.)

Estimation can also be done

• either simultaneously (maximizing the likelihood function that reflects the dynamics of both the conditional mean and the conditional variance)
• or in two steps (first estimate the conditional mean equation, implicitly assuming constant conditional variance, then estimate the conditional variance equation for a given estimated conditional mean).

The problem with two-step estimation is that the first step uses an assumption that is violated in the second step and as such makes the estimators of both steps inefficient and sometimes inconsistent, as discussed in earlier posts (cited e.g. in the OP).

Model selection

Selecting a conditional mean-conditional variance model is not easy because it is a large animal and there are so many choices to make. Selecting sequantially (first the conditional mean model, then the conditional variance model) is suboptimal because the first step depends on assumptions that are violated in the second step, and I am not aware of any theoretical results that guarantee consistent selection or the like, as also discussed in previous posts. Nevertheless, this is sometimes done in practice and even recommended in time series textbooks such as Tsay "Analysis of Financial Time Series" (2005). However, I perceive the recommendation as "a" model selection strategy that is relatively easy, but not necessarily "the best" one.

Among other strategies probably the simplest, although computationally demanding one would be to fix a pool of candidate models (e.g. all submodels of ARMA(4,4)-GARCH(2,2)), estimate them (preferably simultaneously) and select the one with the lowest AIC (if the goal is forecasting) or BIC (if the goal is recovering the "true" model).

Questions 1, 2 and 3

1. This should be clear from the text above.
2. It is possible, but that does not mean it is desirable. As I mentioned, I am not aware of any theoretical results that guarantee consistent selection in the two-step approach. On the other hand, sometimes it makes sense to go for speed and convenience instead of a statistically "clean" approach, and this is what people often do.
3. The model estimated on data until time $t$ will immediately yield a forecast for time $t+1$, because the left hand side of the model leads the right hand side by one time period. If you want to forecast further ahead, do that iteratively: consider the $h-1$-step-ahead forecast as real data and this way "extend your sample" by $h-1$ time points, then do exactly the same as when forecasting 1 step ahead (do not refit the model).
• Thank you for the very thorough answer. So if I am able to write the likelihood functions of ARIMA and GARCH, then an ARIMA(p,i,q)-GARCH(s,r) is just the sum of the two models, similar to how ARMA is the sum of an AR and MA model?
– user124589
Dec 20, 2016 at 6:43
• @rec, it might not be that easy. If you take the likelihood of ARMA as the starting point, you have to include the extra structure of the residuals due to GARCH. I do not think it would be the same as just adding the likelihood together. But as I wrote above, I hope there are some textbooks or lecture notes or maybe software documentation (R, Stata, Matlab) that has an explicit likelihood for ARMA-GARCH. If I were you, I would search carefully online and hope for the best. Dec 20, 2016 at 6:46
• Ah okay, yeah that makes sense since we are modeling the residuals as GARCH I couldn't just add them, the residuals would now need to be represented by the GARCH model's likelihood function. I will dig around for a likelihood function for the ARMA-GARCH model.
– user124589
Dec 20, 2016 at 6:49
• @rec, and if you find the likelihood in some source, please post the link or reference here in the comments, that will be appreciated. Dec 20, 2016 at 6:59
• @rec did you find something while digging up? Apr 9, 2018 at 9:47