0
$\begingroup$

I have recently been doing some analysis regarding forecasting of specific items. I want to know if the findings of my analysis are specific to my data or if it is something that is commonly seen.

I am looking into finished goods finished good forecasts at a sku level and am comparing a statistical forecast from a software package my company uses for forecasting purposes to a moving average of historical demand to calculate value add of the forecasting. At higher levels, say product line, forecast accuracy of the lump sum total pounds of material is very high (say 85%) on the statistical forecast and less so on the moving average (70%), but if I take an average of each sku's accuracy in the product line, the moving average tends to be as good if not better.

For clarification, a sku is a specific item produced at a specific plant. Within my organization, we have 100s, if not thousands of skus. We could have the same item produced at multiple plants, and each of their skus would differ slightly. Basically it's just a unique item identification code.

Is this normal at this level?

Improve this question
$\endgroup$
2
  • $\begingroup$ Since you are doing thousands of parallel forecasts you should look into hierarchical forecasting, see stats.stackexchange.com/questions/31473/… $\endgroup$
    – kjetil b halvorsen ♦
    Jun 3, 2021 at 13:02
  • 1
    $\begingroup$ @kjetilbhalvorsen We are using a forecasting software that does this. Here's the issue. For my particular purposes, we care about maximizing the accuracy of each item within the lowest level of that hierarchy (thus why I am looking at this from this lens). $\endgroup$
    – tfr950
    Jun 3, 2021 at 13:25

1 Answer 1

Reset to default
1
$\begingroup$

This depends on a lot of things but I would say that this is fairly typical. Mostly due to at your lower level (sku level) you probably have more intermittent demand as well as a less stable seasonal signal for more items which makes it harder to do more complex forecasting methods. And, most importantly, you are taking averages across these items where you may actually want to weight the accuracies at this lower level by their volume.

For example, if you have 10 products and 8 of them are low volume and 2 are high volume then you care a lot more about the forecast for the high volume. So you should weigh the higher volume forecasts (which will tend to be higher accuracy for the more advanced methods) more when averaging. Or not, but then you may cater more towards those lower impact skus (which will do better with naive methods more often).

Improve this answer
$\endgroup$
8
  • $\begingroup$ Thanks for the reply! I found the same thing. When looking at the top 20% of skus by volume, they tended to outperform the moving average by ~10%. The only issue is this analysis was done when these forecasts were "locked in". So let's say we are forecasting for the month of June, the forecasts used in this analysis would be "locked in" on May 31. For our purposes, we would need a June forecast from April 15 or so. That leads me to my next question. Do you think that a moving average would be better in this circumstance, even on the high volume items? I can't test this myself sadly. $\endgroup$
    – tfr950
    Jun 3, 2021 at 13:21
  • 1
    $\begingroup$ For such a short horizon it can probably do 'ok' but it just entirely depends on your data and you can't say anything for sure unless you test it. BUT you can have massive misses for holiday events or anything that has that seasonality which for planning would be a pretty big deal. Also, if your large seasonal period such as christmas aren't in the testing dataset it will probably overestimate how much better the moving average is over something that has that seasonality. So I would probably prefer something more complex than the moving average in general. $\endgroup$
    – Tylerr
    Jun 3, 2021 at 15:58
  • 1
    $\begingroup$ Just to add more flavor, the moving average would probably be better for the intermittent time series, it really just comes down to your large volume items typically also having a strong seasonal signal which means that if you show a 'in-season' forecast for a seasonal item that is a moving average it would not be a very 'useful' forecast even if it may be better on average, i.e. the forecast could mean we have more inventory sitting around costing money. What would be better is to try both and use which ever does best on cross validation. So this is a roundabout way of me saying 'it depends' $\endgroup$
    – Tylerr
    Jun 3, 2021 at 16:07
  • 1
    $\begingroup$ That sounds cool but it probably isn't worth the effort and you could probably just do very simple rules to choose without testing. On the other hand you may just want to start ensembling and use both forecasts as well as adding other forecasts and averaging them or combining them with a ML model. That will probably be the best bang for your buck when compared to a ML model for classifying and can even be interpreted via the classification you already have. For example, you can use the probabilities for the 'winning' model as weights to average the model's forecasts together. $\endgroup$
    – Tylerr
    Jun 3, 2021 at 17:03
  • 1
    $\begingroup$ You can look at the FFORMA method for more inspiration: robjhyndman.com/publications/fforma $\endgroup$
    – Tylerr
    Jun 3, 2021 at 17:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.