Does weighted MAPE (wMAPE) provide an accurate estimate of error? I'm trying to use wMAPE to calculate demand standard deviation for future forecasts, however, the wMAPE that we are using is derived from 13 buckets of actuals over a period of 52 buckets of time. If there are 13 weeks of actuals (can be continuous or intermittent) over a horizon of 52 weeks, we calculate the wMAPE based on those actuals. I'm trying to determine if this is the right way of estimating error as the intermittency (demand volatility) seems to be ignored.
I tried finding academic texts on weighted MAPE but I'm not finding anything. Any help will be appreciated.
 A: There is no single "error" that we can estimate using an error metric. An error is whatever we measure. We can definitely measure the error using the MAPE (which may be undefined), the wMAPE, the MSE and so forth. All these errors will be different, and more importantly, each error will be minimized by a different forecast. So the question is one of choosing an error measure that elicits the forecast we want. (Saying "we just want a good forecast" begs the question, because what a "good" forecast is depends on the error measure.)
You may be interested in What are the shortcomings of the Mean Absolute Percentage Error (MAPE)? Although it's about the plain vanilla MAPE, it should be enlightening about the wMAPE.
Since the wMAPE is just the MAE divided by the mean (assuming you use the same definition as Kolassa & Schütz, 2007), it elicits the same forecast as minimizing the MAE - that is, the conditional median. The difference in "optimal" forecasts for different error measures are illustrated here for lognormally distributed data and here for gamma distributed data.
A median forecast may or may not be what you want. I personally have never seen a business process that would be better served by a median forecast than by an expectation or high quantile forecast.
Note in particular that for intermittent demands, if you can't reliably forecast when demand occurs, your conditional median may well be zero, so the wMAPE- or MAE-minimal forecast may well be a flat zero, which is a bit of a problem for people working on intermittent demand forecasting and using the MAE (Kolassa, 2016).
