2
$\begingroup$

I am working on a time series forecasting project in my office.

We have to forecast the demand for 10 products at each store level (no hierarchy). Just at store level is enough..We have 20 stores.So, we have a total of 200 series. We have exogenous variables such as day, week day, month, weather etc as input variables to the series

Now, the problem is each product may exhibit a different demand pattern in each store.For example some parts in store 1 may be fast runners and the same parts in store 3 could be slow runners.

So, my question, is how can I select the model based on time series data? I should already preconfigure the models based on my understanding of demand? For ex - Random Forest, Croston (for intermittent demand), seasonal naive (sporadic or no demand) etc.

So, the only way is even though I know croston is not an appropriate model for constant demand, high runners, code will still run through that model, costing us time and resources?

Is there any elegant existing solution using python which will pick the model based on its understanding of the demand pattern of product in each store?

Or the only solution is to pre-configured the list of models? For ex - let's say I provide 10 model names to the model dictionary, then all the 200 series will be tested for all the 10 models? Is this the only way?

  1. How do time series experts here usually do this in a efficient way?

  2. what is the project methodology that you usually follow to do this sort of project?

$\endgroup$
5
  • $\begingroup$ I see the unevenly-spaced-time-series tag. Is that actually the case, or are you "just" having intermittent-demand? Also, what is your time granularity (days, weeky, months, years)? $\endgroup$ Commented Feb 6 at 12:26
  • $\begingroup$ My data has the purchase date...Since we want to predict the demand for next quarter, we aggregate them at monthly level $\endgroup$
    – The Great
    Commented Feb 7 at 0:04
  • $\begingroup$ I have products that were newly introduced and it has only few samples...whereas some products are there for 2 years with at least 1 unit of purchase every quarter..some products are there for 2 years with 0 purchases in some quarter.. $\endgroup$
    – The Great
    Commented Feb 7 at 0:06
  • $\begingroup$ For a new product that was introduced only 6 months ago, we may have data for 2 years (like other parts). Hence, I chose unevenly spaced time series.. I believe my data would cover all three types - constant demand, intermittent demand and uneven time series (new products) $\endgroup$
    – The Great
    Commented Feb 7 at 0:08
  • $\begingroup$ OK, thanks. I don't quite think that unevenly-spaced-time-series tag makes a lot of sense, your time series are simply of different length. I would suggest you remove that one. I will try to write up an answer later today. $\endgroup$ Commented Feb 7 at 7:24

1 Answer 1

1
$\begingroup$

Your question is rather broad. I will give a few thoughts off the top of my head.

First off, I recommend this forecasting workflow. (Little surprise there, since I wrote it up.) Decide what kind of forecast you want up front. Do you want expectation forecasts or quantiles for setting target stock levels? (As an aside, beware of using the MAE/MASE/wMAPE as an accuracy measure if you have intermittent data.)

You mention that your series are of unequal length. Of course, this should be included in your modeling, so only start modeling once the series is "active". If you can't distinguish no data before the product was activated from zero sales at the beginning of its activity, you could start modeling with the first nonzero sale.

Next, I would define a pool of "reasonable" forecasting models. These should include simple methods like the overall mean and exponential smoothing. Possibly ARIMA, but that is really usually not so good for forecasting accuracy. Perhaps Croston if you have lots of intermittent series (where the overall historical mean is often quite competitive). Consider building custom panel data models that can pool series where you expect common seasonalities or similar, e.g., you might expect similar seasonal patterns for a product in all stores, or for a group of products. Mohammadipour et al. (2012) proposes a simple way of pooling data for seasonality.

And simple is the way to go here. Two years of monthly data is very little, and 10 products in 20 stores is so, too. You will likely not be able to extract a lot of signal, and any complex method will very likely overfit. By all means try a Random Forest (for expectation forecasts - you could try a Quantile Forest for quantile forecasts), but it often does not work very well, and again, you don't have a lot of data.

For the actual method selection, I would go with a straight-up holdout set. Since you are interested in forecasting three months ahead, a logical holdout period would be three months. Fit your models to the first period, holding out the last three months, forecast into those and assess the accuracy using an appropriate error measure: (possibly scaled) (R)MSE for expectation forecasts, pinball losses for quantile forecasts. Possibly use weighting in aggregation, or slice the results in useful ways.

Note that I would not just apply all possible models to all series and pick the one model that performed best on any given series, because likely enough this will not be the best performing model going forward. Instead, use one or more model selection methods and see how this "meta-learner" performed. Tweak this method until it works well, as per the performance on the holdout set. (Don't over-optimize, because you can indeed overfit to the holdout set.)

Do consider averaging models rather than picking one, this very often works very well. Here, consider equal weights, which often work better than optimizing combination weights (the "Forecast Combination Puzzle").

Day of week and weather information will not be very useful for monthly data. Plus, you don't even have reliable weather forecasts for three months ahead, so the main benefit of using weather data is likely to cleanse past sales of the influence of major catastrophes like hurricanes. Then again, you might consider modeling on other granularities, like daily, where information you have on daily level might help, and then reconciling the daily and monthly forecasts (Athanasopoulos et al., 2017).

$\endgroup$
5
  • $\begingroup$ Do you think maintaining the granularity at days level would make any difference? Of course, this will result in a lot of zeroes of no sales (making almost all of my series intermittent). If I aggregate it at month level, may be some of series would have non zero sales throughout.. $\endgroup$
    – The Great
    Commented Feb 8 at 1:29
  • $\begingroup$ Also, I always have this doubt. In time series problems, should we consider too back in the past? For ex, right now, I have 2 years of data chosen for analysis. But I could even choose 4 years of data for analysis. But the problem is lot of things have changed and intuitively it feels that looking too back in the past may only decrease performance (I assume).. because lot of products would have been discontinued etc, so, how do you usually determine what is the appropriate duration of data chosen for prediction at monthly? I ask you because I know you are an expert in this field.. $\endgroup$
    – The Great
    Commented Feb 8 at 1:32
  • $\begingroup$ Re the time granularity: the granularity of the final forecast should be on the level you need for subsequent business decisions. If you can only place an order once per month, you need aggregate monthly forecasts. (Actually, depending on the lead times, you may need multiple cumulative forecasts over different horizons.) But that does not mean you can't use finer grained data and forecasts as intermediate steps. For instance, promotions might happen on daily level, so it makes sense to also consider daily forecasts, and then possibly use an approach like Athanasopoulos to reconcile. $\endgroup$ Commented Feb 8 at 7:19
  • 1
    $\begingroup$ Re the length of history: I would say this is an empirical question. Yes, many things change quickly. But other products have a long lifecycle. For example, tomatoes or milk probably behaves the same way today as 10 years ago. But when it comes to forecasting retail sales around Christmas (where day of week patterns get jumbled up badly), it is hugely valuable to have historical data from multiple years back when Christmas was on the same day of week as in 2024. I would usually use as much data as I can get my hands on, and then use a method that downweights older observations. $\endgroup$ Commented Feb 8 at 7:22
  • $\begingroup$ if you have time, may I seek your help with this post - datascience.stackexchange.com/questions/129729/… $\endgroup$
    – The Great
    Commented Jul 21 at 17:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.