I estimated a couple of GARCH models on the basis of of different estimators for a given sample of data.
Now I have two more data sets, that I want to use to evaluate which of these estimated models gives me the best forecasting accuracy.

For the realized volatility I use the Garman and Klass estimator.

Now my question is:
In the literature almost always the ML estimator is used to estimate the parameters of a GARCH model. However when it comes to the evaluation of the forecast accuracy of GARCH models it seems like Maximum Likelihood is never used as evaluation criterion. I always see the typical loss functions like MSE or MAE as evaluation/ranking criterion. Why is that?

The Maximum Likelihood tells use what the likelihood of the forecasted value being the actual value is, given a particular density function.
To me looking at the likelihood of the forecast beeing the actual value is a much better measure then looking at the MSE. I understand it depends on the use case, but I have NEVER seen it beeing used in a paper on that topic.

So why ML isn't used as evaluation criterion, when it is almost always used to estimate the models?

  • $\begingroup$ Here is a paper I found that actually uses the out of sample likelihood as evaluation criterion: bibliotecadigital.fgv.br/ojs/index.php/bre/article/viewFile/… $\endgroup$ – flxh Apr 4 '17 at 21:01
  • $\begingroup$ A quote from Hastie et al. "The Elements of Statistical Learning" (bottom of p. 221): The log-likelihood can be used as a loss-function for general response densities, such as the Poisson, gamma, exponential, log-normal and others. $\endgroup$ – Richard Hardy Apr 21 '17 at 13:45
  • $\begingroup$ That really says it all. Thank you very much for that quote. $\endgroup$ – flxh Apr 21 '17 at 14:44

I think you are asking why the out of sample likelihood isn't used as the loss function for assessing accuracy. As a matter of fact it is, e.g. See http://ageconsearch.umn.edu/bitstream/18947/1/cp01no01.pdf

  • $\begingroup$ Yes exactly calculating the out of sample likelihood and raking the models by that value. As I said I read a lot of papers on that topic recently, but that method seems quite unpopular. Are their any drawbacks with that method, that I am overseeing? What does that likelihood decribe the probability of the forecast beein the actual/realized value? $\endgroup$ – flxh Apr 4 '17 at 20:47
  • $\begingroup$ Btw I dont have enough reputation to upvote. Please consider upvote my question so I can upvote your answer ;) $\endgroup$ – flxh Apr 4 '17 at 20:48
  • $\begingroup$ A downside is that you need to know the form of the likelihood, for the MSE you simply need to know the mean prediction. The likelihood gives the probability (density in the continuous case) of the data being generated by the model. $\endgroup$ – Richard Redding Apr 4 '17 at 21:18
  • $\begingroup$ Okay as I said in my case I use the MLE for estimating the model, so I already made an asumption about the form of the likelihood. $\endgroup$ – flxh Apr 4 '17 at 21:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.