# Evaluate forecasting ability of GARCH models with RMSE and MAE

I am evaluating different forecasting models and their ability to forecast index volatility during period of market turmoil, using two measurements, Root Mean Square Error and Mean Absolute Error. For the previous evaluated models, ARMA (1,1) as an example, I was able to obtain the residuals and calculate the RMSE quite easily in Stata.

When estimating the GARCH (1,1) model in Stata I am however not able to correctly obtain the residuals in the post estimation procedure, no option to directly obtain the RMSE is available. Perhaps I have misunderstood how one should evaluate the forecasting ability of GARCH models, since the models specifies the conditional variance unlike ARMA which specifies the conditional mean.

Does anyone have a suggestion on how to obtain these evaluation measurements after estimating a GARCH model? And preferably how to do it in Stata.

• The problem with such error metrics is that volatility is not an observable thing, and it depends on the model. Hence the error is not something well defined (compare what with what?). Instead try comparing the likehoods of each model with the test samples that are not used during the estimation procedure. – Cagdas Ozgenc Apr 5 '16 at 8:21
• @CagdasOzgenc: +1, except for a minor quibble: I wouldn't say that the volatility depends on the model (which is something we use to describe a process), but on the data-generating process. – Stephan Kolassa Apr 5 '16 at 8:56
• @StephanKolassa I think what you are saying is a little philosophical. Because there is no single definition of volatility, it can be caused by jumps, quadratic variation, etc. But all these concepts are invented by humans hence take roots from our abstraction, which leads to a model view rather than a true DGP view. True DGP is probably not comprehensible to humans, but only to AI. – Cagdas Ozgenc Apr 5 '16 at 10:44