0
$\begingroup$

Despite the fact that it's necessary to analyze the data first, is it possible to compare accuracies (MAPE/MASE) for differents points of forecast from aggregated data (flow, as in Silvestrini and Veredas)? e.g. monthly provides 12 points for annual, quarter provides 4 points for annual, semester 2 points for annual, annual 1 point for annual. Is it possible to compare between those?

Hyndman article compared different methods (ETS and ARIMA + others) for aggregated data, but on the same basis (annual forecast).

Using ets as an example in AirPassengers:

library(forecast);library(tseries);library(ets)

#data test
a = window(AirPassengers, start = 1954, end = c(1959, 12))
a4 = aggregate(a, nfrequency = 4)
a1 = aggregate(a, nfrequency = 1)

#data train
a_test = window(AirPassengers, start = 1960)
a_test4 = aggregate(a_test, nfrequency =4)
a_test1 = aggregate(a_test, nfrequency =1)

Accuracy results:

> accuracy(forecast(ets(a), h=12), a_test)
                   ME      RMSE       MAE       MPE    MAPE      MASE       ACF1 Theil's U
Training set 1.047724  8.040898  6.313665 0.2072809 1.79992 0.1666607 -0.0448018        NA
Test set     9.264623 19.351491 15.891274 1.7077883 3.29359 0.4194793  0.1677312 0.3992029
> accuracy(forecast(ets(a4), h=4), a_test4)
                     ME     RMSE      MAE        MPE     MAPE      MASE         ACF1 Theil's U
Training set   2.555364 24.51512 18.16731  0.2085802 1.733458 0.1598531  0.006763653        NA
Test set     -18.624399 29.42357 20.30615 -1.4870380 1.601598 0.1786726 -0.237082226  0.029097
> accuracy(forecast(ets(a1), h=1), a_test1)
                    ME     RMSE      MAE       MPE     MAPE      MASE       ACF1
Training set -108.5605 254.1328 195.0527 -3.705815 5.644287 0.4290645 0.00608877
Test set      215.0186 215.0186 215.0186  3.763013 3.763013 0.4729841         NA
> 

May I compare MAPE n MASE from those test sets?

PS: Just using that data as an example, did not analyze it as non-stationary or heterocedasticy

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.