# Alternative to mean absolute percentage error (MAPE)

MAPE metric has problems when the actual value to be predicted is very small. In the extreme when the actual value is 0 then MAPE will be infinity (if the prediction is not exactly 0). What about this solution as illustrated with an example:

Suppose that the actual values are [10, 0, 20] and the corresponding predictions are [13, 2, 16].

Sum of Absolute Errors = 3 + 2 + 4 = 9
Sum of Actual Values = 10 + 0 + 20 = 30
Percentage error = (9/30)*100 = 30


What are the shortcomings of this metric? Also does this metric have a name (I could not find it)? Thanks

Your error is the sum of absolute deviations, divided by the sum of the actuals. If you divide both the numerator and the denominator by the number of samples, we see that it is equal to the mean absolute deviation (or error), divided by the mean of the actuals, or shorter MAD/Mean. We wrote an accessible paper on this (Kolassa & Schütz, Foresight, 2007) and how it can be interpreted as a weighted MAPE, where the weights are just the actuals.

Since you can't influence the actuals in the denominator, minimizing this MAD/Mean or wMAPE is equivalent to minimizing the MAE. Thus, this error measure will be minimized in expectation by the conditional median, not the conditional mean, so using this error measure may give you biased forecasts, especially if your data are conditionally asymmetrically distributed, e.g., lognormally or gamma.

There are many ways how you could normalize error metric, for example, you could divide it by a known constant that serves as a benchmark. What you are suggesting is similar to relative absolute error

$$\mathrm{ RAE} = \frac{ \sum^N_{i=1} | \hat{\theta}_i - \theta_i | } { \sum^N_{i=1} | \overline{\theta} - \theta_i | }$$

but notice two differences:

1. RAE normalized by the distance from the average, rather than sum. In such a case, you compare to a benchmark. With comparing to sum the error would be get smaller as sample size would grow and generally the values may be trickier to interpret than you assume.
2. What if the true values sum to zero? It is totally possible, you just need to have the data with mean equal to zero for that. If they sum to small value you have the same problem as with MAPE.

Every "relative" error metric has some edge cases where it can give strange results or be hard to interpret, that is one of the reasons why people often prefer unnormalized metrics like means absolute error, or mean squared error, etc.