I wanna evaluate a simple GARCH(1,1) model for the conditional variance. Firstly, I understand that the conditional variance is unobserved and that is really the crux of the issue.

Out-of-sample, I have created a loop that re-estimates, for each new observation, and forecasts one period ahead. This works just fine, yet I am unsure of what I should evaluate this against. I first though the squared residuals, but then again? Of what model should these come from? The model with same specifications, but for the whole sample?

Edit: If any one else finds this question, I just wanted to say briefly what I found. When evaluating forecasts of different models, one can use squared returns (if working with stock returns). This might not give correct results (it actually almost surely will, because realized vol is not the same as squared returns), yet the ranking will as if the exercise had been done with the (unobservable) conditional variance.

  • 1
    $\begingroup$ Related questions: 1, 2, 3 and the link in [2] to Patton & Sheppard "Evaluating volatility and correlation forecasts" (2009). $\endgroup$ Nov 25 '18 at 10:17
  • $\begingroup$ If you believe GARCH is a good model for the underlying truth then your best evaluation method is the log likelihood of the GARCH evaluated at out of sample points. $\endgroup$ Nov 25 '18 at 10:26
  • $\begingroup$ @RichardHardy Thanks for the response. You are always helpful. See your name around a lot, and some incredibly helpful comments among them. $\endgroup$ Nov 25 '18 at 14:23
  • $\begingroup$ @CowboyTrader, a more efficient alternative to that would be AIC, hence your suggestion is not "the best". But in any case, what do you mean by "best"? Unless the likelihood coincides with your loss function, how relevant is it to begin with? $\endgroup$ Nov 25 '18 at 14:50
  • $\begingroup$ @RichardHardy OP wants to evaluate performance with out-of-sample data points. AIC is not relevant in this case as the error estimates are not biased. OP hasn't specified a loss function. Log likelihood is a very good surrogate and supported by old and new research in terms of bounding other loss functions. $\endgroup$ Nov 25 '18 at 16:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.