Symmetric AND Weight MAPE Calculation

Question

I'm responsible to forecast a portfolio of consumer products on a monthly basis, and in calculating forecast accuracy, I'm lead to the MAPE (Mean Average Percent Error), which is useful, but has, among its faults, two that I need to address:

1. It's non-symmetric, as the divisor of actuals results in infinity or extremely high numbers if an item is forecasted but not sold (or extremely low sales) in a given time period.
2. It's not weighted, with items of low volume and/or low value having the same influence on the total error as high volume and/or high value items.

The sMAPE (symmetric MAPE) addresses #1 by changing the divisor to the average of teh forecast and the actuals. The WMAPE (weighted MAPE) address #2 by either adding a weighting factor, or, if using volume as the weighting, sums the error before dividing by the sum of the actuals. (I'm leaving it to the reader to either already understand the formulas, or look them up on many available online references).

My question is, I can only find one reference to a symmetric weighted MAPE...a sWMAPE, if you will, that addresses both of the issues. And that one reference simply proposes the formula saying that they couldn't find it documented in the literature.

It seems obvious that this would be a natural evolution of using MAPE -- why isn't it mentioned? I'm using it -- it seems to work quite well!

Answer 1

On the one hand, it's hard to prove a negative, i.e., why people do not use something.

On the other hand, we can look at the MAPE, the sMAPE and the wMAPE.

• The MAPE is easy to interpret, and can be horribly misleading: What are the shortcomings of the Mean Absolute Percentage Error (MAPE)? If you are really looking for unbiased expectation forecasts, then the MAPE will lead you astray, especially if you have an asymmetric conditional distribution or a high coefficient of variation (e.g., when forecasting low volume count data).
• The sMAPE is less easy to interpret, and can be misleading in new ways: if your actual is 0, then any forecast contributes 200%, regardless of what you actually forecast (pointed out by Boylan & Syntetos, 2006, Foresight). It's symmetric in one sense we often don't care all that much about (exchanging the actuals and the forecast leaves the sAPE unchanged), but not in another sense that is usually closer to our notion of "symmetry" (over- and underforecasting by the same amount does not leave the sAPE unchanged): What is "symmetry" in evaluation metrics And it still does not elicit the conditional expectation: Minimizing symmetric mean absolute percentage error (SMAPE)
• The wMAPE is also less easy to interpret than the MAPE, but at least it has an interpretation as a scaled Mean Absolute Error, so it elicits the conditional median - still not the conditional expectation (and the difference gets more relevant for low volume count data, e.g., Kolassa, 2016, IJF), but at least this is a well known functional of central tendency. Scaling error measures is commonly done in forecasting to make the errors comparable between series.

It seems to me like the swMAPE addresses some of the issues of the MAPE, like the division by zero, while being a little less interpretable yet than the sMAPE and the wMAPE. The sAPE's issue of always yielding 200% on a zero actuals is attenuated if you scale by the average of the forecast and the actual.

I (Kolassa, 2020, IJF) would always recommend that you first figure out which functional of the conditional distribution you need, and then choose an error measure that elicits that. If you want an expectation forecast, use a variant of the MSE. If you want a median, use a variant of the MAE. If you want the unnamed-functional-that-the-swMAPE-elicits, then by all means use the swMAPE. Anything else makes little sense. (I have never seen a business process that would be better served by a (-1)-median, which is what the MAPE elicits, than by an expectation forecast.)