9
$\begingroup$

I have a data set of weekly sales for a range of stores (all belonging to one company). I am trying to predict weekly/monthly use of several ingredients in the individual stores. The choice for what type of model to use seems to be between Holt Winter (or state space models more generally) and the class of ARIMA models.

I have made a range of analyses testing which type works best over the entire dataset. That is, I have for example looked at what model describe any given series best and then counted which describes the most series best. This has been done using MSE, MAPE and other measures.

However, I am not sure whether I should simply determine which model is best for any given individual series and then use that. The reasons I haven't done that is because it seems more intuitive to use the same model for all series.

And so my question is, is there any particular theoretical or practical reason, why I would want to choose one method or the other?

Improve this question
$\endgroup$

4 Answers 4

Reset to default
1
$\begingroup$

It is useful to know which time-horizon you care about (a month, a week or a day in advance?), and how much data you have (can you reliably estimate yearly seasonality?).

Personally, I've found ARIMA to be unintuitive and full of traps, and I haven't had much success with it. If you have daily data and care about daily fluctuations, then it would probably the right choice anyway.

But whatever you end up doing, my suggestion is to start with a "simple" regression model, include some yearly seasonality (some cyclic splines), holidays and a trend, ideally set it up with a hierarchical structure like mentioned in another answer. The coefficients will be understandable, it's quite easy to start with simple and expand.

Crossvalidation in timeseries doesn't work, so just create a couple of windows of a reasonable time period (at least two years if you want to have yearly seasonality and are not using a hierarchical model) and evaluate your method by it's forecast of the next week, or month, or year, whatever range you care about. The evaluation is not straightforward (do you care about a forecast in the near future more than a later one?) and should also reflect your business situation.

Improve this answer
$\endgroup$
1
$\begingroup$

I will elaborate one point not mentioned by the other answers.

With many series for different stores/products, there might be competition/substitution effects so you could want to use some form of hierarchical forecasting. Specifically, some product might be possible substitues for other product, leading to negative correlations in sales. There might be seasonality effects common for all/most products, leading to positive correlations. I would start investigation of such effects maybe with a principal components analysis.

If such effects are important (they probably are), some kind of hierarchical prediction could be much better than univariate modeling. Multiple approaches are possible. One way, I used in one project, was first modeling total sales, and then modeling proportions of total sales. That would be top/down, one could also get the other way, start with individual series, and then correcting them if the total gets unrealistic. This is discussed in some other post on this site, like Hierarchical time-series forecasting with complex aggregation constraints or Single prediction vs. summing more granular n-step ahead predictions

There is now even an R package for hierarchical forecasting on CRAN, hts https://CRAN.R-project.org/package=hts Its documentation contains references you should have a look at.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Do your software references detect latent variables like level shifts , local time trends , seasonal pulses , week-of-the-month , holiday effects that distort weekly data and ARIMA structure and possible changes in parameters or error variance over time ? $\endgroup$
    – IrishStat
    Commented Apr 6, 2018 at 19:38
  • $\begingroup$ @IrishStat: I dont know, but such ideas should be possible to combine with hierarchical forecasting? Do autobox support hierarchical forecasting? $\endgroup$
    – kjetil b halvorsen ♦
    Commented Apr 6, 2018 at 19:40
  • 1
    $\begingroup$ yes it does ... it builds model at the child level that incorporate parent data e.g. hourly predictions based upon daily values or item predictions based upon a group value/prediction . At the end the child forecasts have to be reconciled in either a top-down or a bottoms up manner subject to a user selection. $\endgroup$
    – IrishStat
    Commented Apr 6, 2018 at 19:44
  • $\begingroup$ OP posted this more than a year ago, we will see if he is still around ... $\endgroup$
    – kjetil b halvorsen ♦
    Commented Apr 6, 2018 at 19:48
  • $\begingroup$ I suggest that you download AUTOBOX and read the material on Parent to Child to get a full read of the approach to multi-level modelling. $\endgroup$
    – IrishStat
    Commented Apr 6, 2018 at 20:20
0
$\begingroup$

For this type of problem of model selection / hyperparameter-optimization, I would recommend you look into cross-validation approaches. Especially since your primary goal seems to be out-of-sample prediction, you want to be careful about overfitting your training data.

I can't think of a reason why you couldn't have different models for different items, but you may also want to share information across the different models (perhaps some sort of hierarchical setup), which may be more difficult or impossible if the models are incompatible.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Another point is that with many series for different stores/products, there mightbe competition/substitution effects so youcould wantto use soome form of hierarchical forecasting. Search this site. $\endgroup$
    – kjetil b halvorsen ♦
    Commented Oct 23, 2017 at 23:06
0
$\begingroup$

ARIMA models easily incorporate empirically identified pulses ,level shifts and local time trends while incorporating parameter and error variance changes. HW models are a fixed procedure lacking the robustness and adaptability of ARIMA models with Intervention Detection and are bloated by incorporating unnecessary/non-significant parameters. Additionally ARIMA models easily incorporate user-specified causals morphing into Transfer Function Models. Why settle for an assumed model form like HW when diagnostics can lead to better modelling. Better modelling often includes day-of-week effects, holiday effects, monthly effects, weekly effects , day-of-the-month effects et al . You might want to look at this reference http://www.autobox.com/cms/index.php/blog/entry/advantages-and-disadvantages-of-using-monthly-weekly-and-daily-data to more fully understand why you need to be using daily data

Improve this answer
$\endgroup$
9
  • $\begingroup$ Thanks so much for the response. Appreciate that a great deal. However, I am still wondering whether there are any reasons for not just choosing the best model for each individual series (store). $\endgroup$
    – pApaAPPApapapa
    Commented Oct 18, 2016 at 16:29
  • $\begingroup$ Each store should have it's own model with it's own parameters ; to do other would be simply wrong/ presumptive/inefficient. $\endgroup$
    – IrishStat
    Commented Oct 18, 2016 at 16:58
  • 1
    $\begingroup$ It would be iteresting (to me) your justification for this conclusion. ARIMA modelling ( with or without intervention detection) is empirical thus depending upon your software/approach model identification may be the problem while assuming a model form like HW is much less empirical . In the case of a small (in terms of observations) series or a very noisy series , model identification may fail and by chance you are better of with an assumed structure. We often find that for seasonal time series with "few" observations one can do better starting with a HW model and then iterating from there. $\endgroup$
    – IrishStat
    Commented Oct 18, 2016 at 19:59
  • 1
    $\begingroup$ Thanks for the answer. I don't have any conclusion and thus no justification. I was wondering whether there was one which existed. I understand why you favour ARIMA models - I do so myself, however, it is hard to justify to use an ARIMA model over a HW model if the latter has a smaller MASE&MSE and perform better on a test set. $\endgroup$
    – pApaAPPApapapa
    Commented Oct 19, 2016 at 6:17
  • 1
    $\begingroup$ You are conflating two issues. you should be objectively and scientifically computing accuracy for 1 time series from multiple origins. E.G. If you have 60 months of data and are interested in a three period out forecast accuracy. Model 57 values to predict 58,59,60 ; Model 56 values to predict 57,58 and 59 ; ..... Model 46 values to predict 47,48 and 49. Now compute the 12 accuracies and a composite measur. You have taken out the impact of just using 1 origin and have a healthy/robust/origin free measure . Now do this for every store. $\endgroup$
    – IrishStat
    Commented Oct 25, 2016 at 15:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.