Forecasting sales (in units) for thousand of products

Question

I've got into this internship in a retail company and they asked me to think a way to forecast their daily sales (in units) in all their stores (with thousands of skus each one).

At first I thought the simplest way, using some exponential smoothing technique, Holt Winter or something like that, or even sarima or arimax, but after analyzing their data, it turned out it has many products with zero sale days (As far as I remember, arima and exponential smoothing has troubles with many zero values and with more than one seasonal pattern, here I bet we'd have weekly, daily and monthly seasonal pattern), so I'm having doubts about using them.

Another possibility would be use some multivariate regression incorporating variables like day of the week, month, year, holidays, sales off days, etc. but the problem here is that I would have to include the sku and the store as categorical variables with hundreds of levels, is that correct?

Another possibility more complex would be to use the same regression variables with neural networks or random forest regression with boosting, but again, what to do about that huge amount of levels?

I'm Also considering take week as the deepest level in the forecast, and then using some weights to distribute if necessary (daily). Finally, one approach I didn't mention was to make some variables based on the autocorrelation function to try to model the period dependency structure such as sales of the week of one month ago, one year ago and probably the slope in those periods, then using some regression model adjustment.

So here I am, asking for some guidance, maybe a book, an idea, an example. Anything would be very appreciated.

ps: Sorry, english is not my first language.

Answer 1

A simple approach would be to use a regression with dummies to account for seasonality and promotions (if your retailer has them). I would recommend negative binomial regression or Poisson regression because of the slow mover problem you have identified.

You can do this on weekly level and distribute the forecast to days using weights, as you propose, or work directly on daily level with weekday dummies. The latter would be easier if your retailer has promotions whose length does not exactly coincide with calendar weeks.

You can build one giant model with lots of dummies for products and stores (or, more sophisticatedly, a hierarchical model), or go the simpler route of separate models per prod-loc. If you have many parameters and few observations, some kind of regularization may be helpful.

Previous threads may be useful, in particular this one. I like to believe that an article I wrote (Kolassa, 2016, International Journal of Forecasting) might be enlightening.