First I would like to thank R people & package developpers for making available such a collection of great tools.

My question is about forecasts reconciliation, as I find huge differences between the original forecasts and the reconciliated ones.

I have a set of grouped/hierarchical time series representing sales, like this:

Total: -> Business units: BU --> Product categories: CAT ---> Manufacturers: MNF ----> Inventory type: INV -> Sales area: AREA --> Customer group: GROUP

So a bottom serie is like this: BU/CAT/MNF/INV/AREA/GROUP. EG: MOBILITY_SMARTPHONE_MNFxxx_LOCAL_B2C_GROUPyyy. At the end there are about 13.000 series of monthly data, since jan 2016. In principle, the forecast horizon is 12 months. Series might show different kind of seasonality (monhtly, quarterly), some are declining, other are increasing, some are intermittent etc.

My approach is:

  • as series are very different from each other, I've chosen a set of different forecasting methods too - basically from Forecast package: tslm, auto arima, snaive, ETS, baggedETS, theta, tbats (for double seasonality: 12 and 3)... And some combination of these (a simple mean after eliminating possible outliers).
  • series are divided into 4 groups depending of their weight over the last 6 months: "heavy", "light", "zero" and "new" ("new" might be "heavy" series, but with very few observations, eg. only the last 3 months) - this allows me to "preselect" a convenient subset of methods (and save calculation time!)
  • I use a training/test sets approach, to select the best method for each serie, vs last 6 months real data.
  • Then I forecast each serie, at every level, using the previously selected method
  • And finally I use the combinef function (with LU algorithm), to reconciliate all these forecasts.

This works perfectly well! The real sales for november were 91M€ while my reconciliated total forecast was 92M€ (and the original one was 94M€). Too beautiful to be true, maybe!

The "issue" I'm facing now is that the difference between the original and reconciliated forecasts is quite big (I mean, more than expected: to me, say 5% would be OK), at any level. Some examples of forecasts for this month of December:

  • for the total: 114M€ (reconciliated) vs 135M€ (original)!!!
  • for a single manufacturer: 5,7M€ vs 1,3M€!!!

So I would really appreciate:

  • your opinion about the approach I've chosen
  • suggestions about how to improve it, by minimizing the differences reconciliated vs original forecasts,

a/ Maybe, by using a different algorithm than LU?

b/ What about the "optimal reconciliation" Hyndman's method? Would that make sense to use it with the combinef function? How?

c/ Or should be aware of the weights option of the function? How to calculate them automatically?

d/ Or has it simply got to do with the data? Would some kind of initial detection be convenient? But what would you think I should be looking for?

e/ From another point of view, I'm willing to test M4MetaLearning method, in order to improve the original forecasts. Do you think its use could substitute the all process i've set up?

Thank you very much in advance, Alain.

  • $\begingroup$ First of all, kudos for setting all these up. HTS applications are far from trivial. Second, as you correctly note LU is not your only option. For example you find find that a bottom-up (BU) approach also helps to get "reasonable looking" results; one of the "surprising" findings for me from the RJH et al. "Optimal combination forecast" paper was that the BU approach did so well! Third, notice also that if we average the estimates we get from any valid reconciliation method the estimates will still be reconciled so one might want to explore the idea of combining the reconciled forecasts. $\endgroup$ – usεr11852 Dec 9 '18 at 23:04
  • $\begingroup$ Thank you for these good ideas/suggestions, user11852... I'll have a try... Today I've noticed too that some "devil" was hidden in the data: kind of a "strange" serie, with a really anomalous value for last november... So I'll also have to improve the initial grouping of the series. Hopefully all that will improve the results!!! Thanks again! $\endgroup$ – Alain A Dec 10 '18 at 18:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.