1
$\begingroup$

I have four years of monthly data, but I want to use three years to test the forecast of a model which uses only one year of historical data in a rolling way. For example:

  • historical data from Jan-2016 to Dec-2016 will be used to forecast Jan-2017 to Dec-2017
  • but then Feb-2016 to Jan-2017 data will be used to forecast Feb-2017 to Jan-2018.

Can I still use MASE to test the accuracy of this forecast?

$\endgroup$
3
$\begingroup$

Rolling evaluation of forecast accuracy is an extremely good idea.

You can use the or any other accuracy measure like the or to evaluate rolling or non-rolling forecasts - your setup doesn't matter for your choice of evaluation measure.

Indeed, if you look into the academic literature, e.g., the International Journal of Forecasting, you will see many examples of rolling evaluations using the MASE or other error measures.

$\endgroup$
  • $\begingroup$ Do I assume that even though I'm using 12 months to determine a 1 month forecast, the error is based on the full 12 month forecast and not just the one that is not being over-written? That's what's confusing me. $\endgroup$ – Angus Jan 31 '18 at 18:26
  • $\begingroup$ That will depend on how you code your error measure. If you use only a single forecast (e.g., only Jan-2017 in the first case), you get an "ASE", no "Mean" involved. But if you then average these single errors over your rolling setup, you again get a MASE. Or you can do "averages of averages", depending on what forecast horizon you are really interested in. $\endgroup$ – Stephan Kolassa Jan 31 '18 at 19:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.