I've been trying to assess the best accuracy measure, especially for intermittent demand. While I have found scaled measures such as RMSSE and MASE to be really good measure I find that their interpretation in too difficult for the business world.

The MAD/Mean ratio (or WMAPE) seems to be a good fit overall, as it rarely divides by zero and puts equal weights on positive / negative forecasts.

What are some disadvantages of the MAD/Mean ratio (are there any)?


One major problem with the MAD/Mean especially in an intermittent demand forecasting context is the following: the MAD will be minimized in expectation by the median of the future distribution. For intermittent data, this may easily be zero. So the "best" forecast, in terms of the MAD/Mean, may be a flat zero line. This is usually not what you want, or what the consumers of your forecast expect.

This effect is well known in some communities, and less so in others. For instance, some intermittent forecasting papers had to argue somewhat contrivedly that their approach was really better than a flat zero, although their error measures would have preferred the zero... I drew attention to this in Kolassa (2016, IJF) in the context of (intermittent) retail sales forecasting, but again, I was certainly not the first to note this.

Thus, I would argue (Kolassa, 2020, IJF) that your choice of error function should be guided by what functional of the future distribution you wish to elicit. If you want unbiased expectation forecasts, use the MSE, or possibly a root or scaled version. If you want a quantile forecast, use an appropriate pinball loss. And so forth.

Related (since the MAD/Mean is just a weighted MAPE in the non-intermittent case): What are the shortcomings of the Mean Absolute Percentage Error (MAPE)?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.