Best performance evaluation measure for GARCH model? [duplicate]

Question

I want to compare the performance of various volatility models like GARCH, eGARCH, and gjrGARCH from actual volatility (computed using high frequency data). I found three common performance evaluation measures used profoundly in literature, i.e. Root Mean Square Error (RMSE), Mean Absolute Error (MAE), and Mean Absolute Percentage Error (MAPE). But I found conflicting results from the above three criteria, like I found RMSE is minimum for GARCH model, MAE is minimum for eGARCH model, and MAPE is minimum for gjrGARCH model.

I want to know which one is the best measure for evaluating performance and which results should I report? If all three are equally valid then how can I interpret the results? Is there any other way to compare the performance of various volatility models other than performance measures?

Answer 1

What is best depends on your objective or loss function.

For example, if you incur loss that is directly proportional to the absolute deviation of the forecast from the realization, the relevant measure would be mean absolute error (MAE). But perhaps you do not care as much about overpredicting as underpredicting? That would imply an asymmetric loss function. Or perhaps you do not care as long as the loss is smaller than some threshold value? Then you would use a threshold loss function.

In conclusion, which measure to use and to report will depend on your objective or loss function. Just make sure you are assessing out-of-sample performance as in-sample-performance may be superficially good due to overfitting.