2
$\begingroup$

I'm learning now some metrics of goodness about time series forecasting, using the forecast package, but I'm stuck in something that surely I've not understood well. I have two vectors, of real data, and forecast data, and I'd like to calculate MAPE.

real <- c(1,2,3,0)
pred <- c(2,3,1,0) 
accuracy(pred,real)
         ME     RMSE MAE       MPE     MAPE
Test set  0 1.224745   1 -27.77778 72.22222

So I decided to try to calculate myself the MAPE, using the docs, so:

mean(abs((real-pred)/real*100))
[1] NaN

And clearly I got NaN, because I have a (0-0)/0.

My question is: am I calculating wrong manually the MAPE, or the function has a kind of na.rm = TRUE?

$\endgroup$
4
$\begingroup$

You are correct that this should not really happen. And no, you are not doing anything wrong in calculating the MAPE.

(Note that accuracy() expects its first argument to be the forecast, and the second one to be the actuals.)

What seems to be happening is that accuracy() removes all data points where both forecasts and actuals are zero - but only for the MPE and the MAPE - and then calculates the KPIs for the remaining data points. The other KPIs are calculated with all data.

> library(forecast)
> 
> accuracy(c(1,2,3,0),c(2,3,1,0))
         ME     RMSE MAE       MPE     MAPE
Test set  0 1.224745   1 -38.88889 94.44444
> 
> accuracy(c(1,2,3),c(2,3,1))
         ME     RMSE      MAE       MPE     MAPE
Test set  0 1.414214 1.333333 -38.88889 94.44444

If we have a non-zero forecast corresponding to a zero actual, we get the correct Inf:

> accuracy(c(1,2,3,1),c(2,3,1,0))
            ME     RMSE  MAE  MPE MAPE
Test set -0.25 1.322876 1.25 -Inf  Inf

I think the behavior makes a kind of sense, but it should be documented in the help page for accuracy(). Perhaps you want to create a ticket and request that this be documented, linking to this thread? For the record, this applies to forecast version 8.9 under R version 3.6.1.

$\endgroup$
  • 1
    $\begingroup$ Thanks a lot! As soon as possible I'll create a ticket about it (I've not a github profile, but it's on my to do list), also edited the question with the right order. $\endgroup$ – s_t Sep 26 at 9:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.