2
$\begingroup$

For purely educational reasons I'm currently trying to fit different types of GARCH models, varying on the order parameters as well as flavor (standard, eGARCH, iGARCH, GJR-GARCH) and different distribution assumptions.
Is it possible to compare them by their log-likelihood?
If not, why and how if even possible do I approach this?

$\endgroup$
2
  • $\begingroup$ Your second question Can I compare log-likelihood between ARIMA and GARCH models? is a duplicate of existing threads, and the answer to it can be found here and here. Since the question makes the post too broad and has already been answered before, I have removed it. $\endgroup$ Jul 1 '20 at 7:18
  • $\begingroup$ What question(s) are you trying to answer by your comparison? I am asking to see if you really need to compare the likelihoods. Are you aware that a richer model will necessarily have a higher likelihood when the models are nested and is likely to have it when they are not nested? That is why we use information criteria such as AIC for comparison of likelihoods adjusted for their degrees of overfitting. See the 4 threads I have linked under the answer of CasusBelli. $\endgroup$ Jul 1 '20 at 7:56
1
$\begingroup$

To compare fit among the models, I would suggest looking at a fit metric, such as RMSE or MAE (depending on how severely you want to punish deviations) relative to empirical data. Now, if you're trying to determine the optimal arguments (p, d, q) for a time-series model, I would look at the AIC or BIC to balance parsimony with fit.

In short: AIC or BIC within each model class; RMSE or MAE across model classes.

$\endgroup$
4
  • $\begingroup$ Is there anything wrong with AIC across classes? I think this questions has been discussed multiple times here on CV, and my takeaway is that you can compare AICs regardless of whether the models are in the same class. Also, likelihood and metrics like RMSE and MAE do not have the same function/properties, so exchanging one for another is not innocuous and should be justified. What is gained and what is lost when you substitute the likelihood by RMSE or MAE for model comparison? $\endgroup$ Jun 30 '20 at 21:03
  • $\begingroup$ The models are fundamentally different; you are essentially no longer comparing apples to apples. You can use AIC / BIC to compare f(a) to f(b) but not f(a) to g(a). This is from what I've gathered reading on the net; I'll be the first to admit that my experience with GARCH / ARIMA is primarily through application -- not theory -- so I'm afraid I can't provide a formal proof to back my position. $\endgroup$
    – CasusBelli
    Jul 1 '20 at 0:45
  • $\begingroup$ The models are not fundamentally different. Save for the distributional assumption, they are all nested in a sufficiently flexible GARCH specification, fit by the same method (maximum likelihood) on the exact same data set (unlike, say, conditioning on different sets of initial observations such as in AR models fit by conditional least squares). Different distributional assumptions is the only point that may be troubling, but from what I have read here on CV and the links provided, I maintain that their AIC and BIC values are directly comparable. $\endgroup$ Jul 1 '20 at 6:30
  • 1
    $\begingroup$ Relevant CV threads: 1 (includes some good references and Akaike's own quote on the topic), 2 (a dedicated question), 3 (a cautious stance), 4 (includes a note from SAS in favour of comparison of nonnested models). The first thread should be sufficient on its own. $\endgroup$ Jul 1 '20 at 7:50

Not the answer you're looking for? Browse other questions tagged or ask your own question.