I need to compare the forecasting ability of different specifications of the ARMA-GARCH model. I would like to compare the model by valuating for each model in-sample forecast and out-of-sample forecast performance.
I know in-sample-forecasts are those generated for the same set of data that was used to estimate model's parameters, whereas out-of-sample forecasts are those generated for the set of data not used to estimate model's parameters.
Since for out-of-sample forecast the multi-step-ahead foecasts could not be ideal as the forecasting horizon increases, one away around this problem is to use a recursive or rolling window. In the latter cases, recursive or rolling, I would re-estimate the model's parameters several times, so I don't have just one estimated model, and I guess estimated parameters will be varying a bit between each estimation.
Hence my question is: how do I evaluate the in-sample forecast given that I estimate the model several times? Could I estimate, for in-sample fit evaluation, the models using all the sample data?
My idea was:
- Estimate the models using all the data.
- Compare the in-sample performace using information criteria as AIC, BIC and AICc.
- Rank the models.
For out-of-sample forecast I would
- Divide the data set into two subsamples and then run a rolling window forecast.
- Then evaluate the forecast performance using MSE, MAE, DAC and Value-at-risk violations.
- Rank the models.
Finally, I would compare the ranking from in-sample forecast with the ranking from out-of-sample forecast and make my conclusions.
Does it make sense?
