First I would like to thank R people & package developpers for making available such a collection of great tools.
My question is about forecasts reconciliation, as I find huge differences between the original forecasts and the reconciliated ones.
I have a set of grouped/hierarchical time series representing sales, like this:
Total: -> Business units: BU --> Product categories: CAT ---> Manufacturers: MNF ----> Inventory type: INV -> Sales area: AREA --> Customer group: GROUP
So a bottom serie is like this: BU/CAT/MNF/INV/AREA/GROUP. EG: MOBILITY_SMARTPHONE_MNFxxx_LOCAL_B2C_GROUPyyy. At the end there are about 13.000 series of monthly data, since jan 2016. In principle, the forecast horizon is 12 months. Series might show different kind of seasonality (monhtly, quarterly), some are declining, other are increasing, some are intermittent etc.
My approach is:
- as series are very different from each other, I've chosen a set of different forecasting methods too - basically from Forecast package: tslm, auto arima, snaive, ETS, baggedETS, theta, tbats (for double seasonality: 12 and 3)... And some combination of these (a simple mean after eliminating possible outliers).
- series are divided into 4 groups depending of their weight over the last 6 months: "heavy", "light", "zero" and "new" ("new" might be "heavy" series, but with very few observations, eg. only the last 3 months) - this allows me to "preselect" a convenient subset of methods (and save calculation time!)
- I use a training/test sets approach, to select the best method for each serie, vs last 6 months real data.
- Then I forecast each serie, at every level, using the previously selected method
- And finally I use the combinef function (with LU algorithm), to reconciliate all these forecasts.
This works perfectly well! The real sales for november were 91M€ while my reconciliated total forecast was 92M€ (and the original one was 94M€). Too beautiful to be true, maybe!
The "issue" I'm facing now is that the difference between the original and reconciliated forecasts is quite big (I mean, more than expected: to me, say 5% would be OK), at any level. Some examples of forecasts for this month of December:
- for the total: 114M€ (reconciliated) vs 135M€ (original)!!!
- for a single manufacturer: 5,7M€ vs 1,3M€!!!
So I would really appreciate:
- your opinion about the approach I've chosen
- suggestions about how to improve it, by minimizing the differences reconciliated vs original forecasts,
a/ Maybe, by using a different algorithm than LU?
b/ What about the "optimal reconciliation" Hyndman's method? Would that make sense to use it with the combinef function? How?
c/ Or should be aware of the weights option of the function? How to calculate them automatically?
d/ Or has it simply got to do with the data? Would some kind of initial detection be convenient? But what would you think I should be looking for?
e/ From another point of view, I'm willing to test M4MetaLearning method, in order to improve the original forecasts. Do you think its use could substitute the all process i've set up?
Thank you very much in advance, Alain.