0
$\begingroup$

I have a question about the role of the MAPE in the ARIMA model optimization.

For a daily time series I have found that the best model (using the Box-Jenkins approach) is an ARIMA(7,0,7)(0,0,0). If I check the ACF and PACF of the model residuals, I see that there is no more information I can extract; see the right panel of the picture below.

However, the MAPE value is as high as 18209.16. To me this sounds too high according to its definition.

Have I missed something? Can I accept this ARIMA model even if it has such a high MAPE?

By the way, I am getting similar results if I use auto.arima; MAPE is always big.

enter image description here

$\endgroup$
0
$\begingroup$

Citing Rob J. Hyndman "Measuring forecast accuracy" (2014),

Measures based on percentage errors have the disadvantage of being infinite or undefined if $y_t = 0$ for any observation in the test set, and having extreme values when any $y_t$ is close to zero.

Since the other accuracy measures seem OK (e.g. MASE well below 1), it might be the specifics of the data rather than a faulty model that is producing the very high MAPE.

Also note that (again citing from the same source)

[a]nother problem with percentage errors <...> is that they assume a scale based on quantity. If $y_t$ is measured in dollars, or kilograms, or some other quantity, percentages make sense. On the other hand, a percentage error makes no sense when measuring the accuracy of temperature forecasts on the Fahrenheit or Celsius scales, because these are not measuring a quantity.

So perhaps you could discard MAPE right at the start based on what your raw data is measuring.

$\endgroup$
0
$\begingroup$

Daily data is often better modeled with a mixture of ARIMA and deterministic effects. Day-of-the-week , month-of-the-year , holiday effects etc. while accounting for pulses/level shifts/time trends. Look at http://www.autobox.com/cms/index.php/afs-university/intro-to-forecasting/doc_download/53-capabilities-presentation slide 47 for an example of this.

$\endgroup$
  • $\begingroup$ Thanks for your answer. When you talk about deterministic effects do you means that I manage as you have described here? [link] (stats.stackexchange.com/questions/48921/…) (similar to page 20). So this should means that I have to clean the data not only using the IQR approach but also considering some "peak" in the data, right ? $\endgroup$ – DP78 Aug 15 '16 at 21:16
  • $\begingroup$ The software should handle the cleaning of he data reflecting not only pulses but seasonal pulses, level shifts and time trends ..... $\endgroup$ – IrishStat Aug 15 '16 at 22:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.