Why forecast::accuracy() mape is working with 0/0?

Question

I'm learning now some metrics of goodness about time series forecasting, using the forecast package, but I'm stuck in something that surely I've not understood well. I have two vectors, of real data, and forecast data, and I'd like to calculate MAPE.

real <- c(1,2,3,0)
pred <- c(2,3,1,0) 
accuracy(pred,real)
         ME     RMSE MAE       MPE     MAPE
Test set  0 1.224745   1 -27.77778 72.22222

So I decided to try to calculate myself the MAPE, using the docs, so:

mean(abs((real-pred)/real*100))
[1] NaN

And clearly I got NaN, because I have a (0-0)/0.

My question is: am I calculating wrong manually the MAPE, or the function has a kind of na.rm = TRUE?

Answer 1

You are correct that this should not really happen. And no, you are not doing anything wrong in calculating the MAPE.

(Note that accuracy() expects its first argument to be the forecast, and the second one to be the actuals.)

What seems to be happening is that accuracy() removes all data points where both forecasts and actuals are zero - but only for the MPE and the MAPE - and then calculates the KPIs for the remaining data points. The other KPIs are calculated with all data.

> library(forecast)
> 
> accuracy(c(1,2,3,0),c(2,3,1,0))
         ME     RMSE MAE       MPE     MAPE
Test set  0 1.224745   1 -38.88889 94.44444
> 
> accuracy(c(1,2,3),c(2,3,1))
         ME     RMSE      MAE       MPE     MAPE
Test set  0 1.414214 1.333333 -38.88889 94.44444

If we have a non-zero forecast corresponding to a zero actual, we get the correct Inf:

> accuracy(c(1,2,3,1),c(2,3,1,0))
            ME     RMSE  MAE  MPE MAPE
Test set -0.25 1.322876 1.25 -Inf  Inf

I think the behavior makes a kind of sense, but it should be documented in the help page for accuracy(). Perhaps you want to create a ticket and request that this be documented, linking to this thread? For the record, this applies to forecast version 8.9 under R version 3.6.1.