1
$\begingroup$

I have 24 months of data of which the last 12 months I'm using to test the forecast for a rolling forecast with a 12 month window. Let's say I want to test the accuracy of the forecast at horizon 6 (point 18). How would I go about doing that? I've seen error measures that use h=6 and every point to the end of the data set.

I forgot to mention, the forecasts are based on taking 12 months of data and forecasting the 13th month. Then the window just moves ahead by one month. Do I assume that I can only look at h=1 forecast accuracy?

$\endgroup$
1
$\begingroup$

at horizon 6 you only have 6 out-of-sample values (future values) to measure your forecast against. I would simple predict 1 period out from each of the 12 origins i.e. point 12,13,....23 . Restimate the model/parameters at each of the 12 origins to reflect typical updating . It would be naive to simply use the model/parameters based upon the first 12 values alone.

In this way you get 12 estimates of a 1 period out forecast error. Now if you wished to use a 2 period horizon .. you would have 11 estimates .

$\endgroup$
  • $\begingroup$ So I assume I calculate forecast accuracy measures, i.e., MSE, MASE etc., on the 12 - 1-step ahead forecasts? $\endgroup$ – Angus May 17 '18 at 20:00
  • $\begingroup$ yes ... and also the Symmetric Mape which is now very popular, $\endgroup$ – IrishStat May 17 '18 at 20:08
  • $\begingroup$ I thought that was discourage when dealing with values close to zero. $\endgroup$ – Angus May 17 '18 at 20:13
  • $\begingroup$ I also saw that caveat at wikopedia .... en.wikipedia.org/wiki/Symmetric_mean_absolute_percentage_error $\endgroup$ – IrishStat May 17 '18 at 20:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.