1
$\begingroup$

I estimated a couple of GARCH models on the basis of of different estimators for a given sample of data.
Now I have two more data sets, that I want to use to evaluate which of these estimated models gives me the best forecasting accuracy.

For the realized volatility I use the Garman and Klass estimator.

Now my question is:
In the literature almost always the ML estimator is used to estimate the parameters of a GARCH model. However when it comes to the evaluation of the forecast accuracy of GARCH models it seems like Maximum Likelihood is never used as evaluation criterion. I always see the typical loss functions like MSE or MAE as evaluation/ranking criterion. Why is that?

The Maximum Likelihood tells use what the likelihood of the forecasted value being the actual value is, given a particular density function.
To me looking at the likelihood of the forecast beeing the actual value is a much better measure then looking at the MSE. I understand it depends on the use case, but I have NEVER seen it beeing used in a paper on that topic.

So why ML isn't used as evaluation criterion, when it is almost always used to estimate the models?

Improve this question
$\endgroup$
3
  • $\begingroup$ Here is a paper I found that actually uses the out of sample likelihood as evaluation criterion: bibliotecadigital.fgv.br/ojs/index.php/bre/article/viewFile/… $\endgroup$
    – flxh
    Apr 4, 2017 at 21:01
  • $\begingroup$ A quote from Hastie et al. "The Elements of Statistical Learning" (bottom of p. 221): The log-likelihood can be used as a loss-function for general response densities, such as the Poisson, gamma, exponential, log-normal and others. $\endgroup$
    – Richard Hardy
    Apr 21, 2017 at 13:45
  • $\begingroup$ That really says it all. Thank you very much for that quote. $\endgroup$
    – flxh
    Apr 21, 2017 at 14:44

1 Answer 1

Reset to default
0
$\begingroup$

I think you are asking why the out of sample likelihood isn't used as the loss function for assessing accuracy. As a matter of fact it is, e.g. See http://ageconsearch.umn.edu/bitstream/18947/1/cp01no01.pdf

Improve this answer
$\endgroup$
4
  • $\begingroup$ Yes exactly calculating the out of sample likelihood and raking the models by that value. As I said I read a lot of papers on that topic recently, but that method seems quite unpopular. Are their any drawbacks with that method, that I am overseeing? What does that likelihood decribe the probability of the forecast beein the actual/realized value? $\endgroup$
    – flxh
    Apr 4, 2017 at 20:47
  • $\begingroup$ Btw I dont have enough reputation to upvote. Please consider upvote my question so I can upvote your answer ;) $\endgroup$
    – flxh
    Apr 4, 2017 at 20:48
  • $\begingroup$ A downside is that you need to know the form of the likelihood, for the MSE you simply need to know the mean prediction. The likelihood gives the probability (density in the continuous case) of the data being generated by the model. $\endgroup$
    – Richard Redding
    Apr 4, 2017 at 21:18
  • $\begingroup$ Okay as I said in my case I use the MLE for estimating the model, so I already made an asumption about the form of the likelihood. $\endgroup$
    – flxh
    Apr 4, 2017 at 21:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.