6
$\begingroup$

I have a data set of weekly sales for a range of stores (all belonging to one company). I am trying to predict weekly/monthly use of several ingredients in the individual stores. The choice for what type of model to use seems to be between Holt Winter (or state space models more generally) and the class of ARIMA models.

I have made a range of analyses testing which type works best over the entire dataset. That is, I have for example looked at what model describe any given series best and then counted which describes the most series best. This has been done using MSE, MAPE and other measures.

However, I am not sure whether I should simply determine which model is best for any given individual series and then use that. The reasons I haven't done that is because it seems more intuitive to use the same model for all series.

And so my question is, is there any particular theoretical or practical reason, why I would want to choose one method or the other?

$\endgroup$
0
$\begingroup$

For this type of problem of model selection / hyperparameter-optimization, I would recommend you look into cross-validation approaches. Especially since your primary goal seems to be out-of-sample prediction, you want to be careful about overfitting your training data.

I can't think of a reason why you couldn't have different models for different items, but you may also want to share information across the different models (perhaps some sort of hierarchical setup), which may be more difficult or impossible if the models are incompatible.

$\endgroup$
  • $\begingroup$ Another point is that with many series for different stores/products, there mightbe competition/substitution effects so youcould wantto use soome form of hierarchical forecasting. Search this site. $\endgroup$ – kjetil b halvorsen Oct 23 '17 at 23:06
0
$\begingroup$

ARIMA models easily incorporate empirically identified pulses ,level shifts and local time trends while incorporating parameter and error variance changes. HW models are a fixed procedure lacking the robustness and adaptability of ARIMA models with Intervention Detection and are bloated by incorporating unnecessary/non-significant parameters. Additionally ARIMA models easily incorporate user-specified causals morphing into Transfer Function Models. Why settle for an assumed model form like HW when diagnostics can lead to better modelling. Better modelling often includes day-of-week effects, holiday effects, monthly effects, weekly effects , day-of-the-month effects et al . You might want to look at this reference http://www.autobox.com/cms/index.php/blog/entry/advantages-and-disadvantages-of-using-monthly-weekly-and-daily-data to more fully understand why you need to be using daily data

$\endgroup$
  • $\begingroup$ Thanks so much for the response. Appreciate that a great deal. However, I am still wondering whether there are any reasons for not just choosing the best model for each individual series (store). $\endgroup$ – pApaAPPApapapa Oct 18 '16 at 16:29
  • $\begingroup$ Each store should have it's own model with it's own parameters ; to do other would be simply wrong/ presumptive/inefficient. $\endgroup$ – IrishStat Oct 18 '16 at 16:58
  • 1
    $\begingroup$ It would be iteresting (to me) your justification for this conclusion. ARIMA modelling ( with or without intervention detection) is empirical thus depending upon your software/approach model identification may be the problem while assuming a model form like HW is much less empirical . In the case of a small (in terms of observations) series or a very noisy series , model identification may fail and by chance you are better of with an assumed structure. We often find that for seasonal time series with "few" observations one can do better starting with a HW model and then iterating from there. $\endgroup$ – IrishStat Oct 18 '16 at 19:59
  • 1
    $\begingroup$ Thanks for the answer. I don't have any conclusion and thus no justification. I was wondering whether there was one which existed. I understand why you favour ARIMA models - I do so myself, however, it is hard to justify to use an ARIMA model over a HW model if the latter has a smaller MASE&MSE and perform better on a test set. $\endgroup$ – pApaAPPApapapa Oct 19 '16 at 6:17
  • 1
    $\begingroup$ You are conflating two issues. you should be objectively and scientifically computing accuracy for 1 time series from multiple origins. E.G. If you have 60 months of data and are interested in a three period out forecast accuracy. Model 57 values to predict 58,59,60 ; Model 56 values to predict 57,58 and 59 ; ..... Model 46 values to predict 47,48 and 49. Now compute the 12 accuracies and a composite measur. You have taken out the impact of just using 1 origin and have a healthy/robust/origin free measure . Now do this for every store. $\endgroup$ – IrishStat Oct 25 '16 at 15:23
0
$\begingroup$

It is useful to know which time-horizon you care about (a month, a week or a day in advance?), and how much data you have (can you reliably estimate yearly seasonality?).

Personally, I've found ARIMA to be unintuitive and full of traps, and I haven't had much success with it. If you have daily data and care about daily fluctuations, then it would probably the right choice anyway.

But whatever you end up doing, my suggestion is to start with a "simple" regression model, include some yearly seasonality (some cyclic splines), holidays and a trend, ideally set it up with a hierarchical structure like mentioned in another answer. The coefficients will be understandable, it's quite easy to start with simple and expand.

Crossvalidation in timeseries doesn't work, so just create a couple of windows of a reasonable time period (at least two years if you want to have yearly seasonality and are not using a hierarchical model) and evaluate your method by it's forecast of the next week, or month, or year, whatever range you care about. The evaluation is not straightforward (do you care about a forecast in the near future more than a later one?) and should also reflect your business situation.

$\endgroup$
0
$\begingroup$

I will elaborate one point not mentioned by the other answers.

With many series for different stores/products, there might be competition/substitution effects so you could want to use some form of hierarchical forecasting. Specifically, some product might be possible substitues for other product, leading to negative correlations in sales. There might be seasonality effects common for all/most products, leading to positive correlations. I would start investigation of such effects maybe with a principal components analysis.

If such effects are important (they probably are), some kind of hierarchical prediction could be much better than univariate modeling. Multiple approaches are possible. One way, I used in one project, was first modeling total sales, and then modeling proportions of total sales. That would be top/down, one could also get the other way, start with individual series, and then correcting them if the total gets unrealistic. This is discussed in some other post on this site, like Hierarchical time-series forecasting with complex aggregation constraints or Single prediction vs. summing more granular n-step ahead predictions

There is now even an R package for hierarchical forecasting on CRAN, hts https://CRAN.R-project.org/package=hts Its documentation contains references you should have a look at.

$\endgroup$
  • $\begingroup$ Do your software references detect latent variables like level shifts , local time trends , seasonal pulses , week-of-the-month , holiday effects that distort weekly data and ARIMA structure and possible changes in parameters or error variance over time ? $\endgroup$ – IrishStat Apr 6 '18 at 19:38
  • $\begingroup$ @IrishStat: I dont know, but such ideas should be possible to combine with hierarchical forecasting? Do autobox support hierarchical forecasting? $\endgroup$ – kjetil b halvorsen Apr 6 '18 at 19:40
  • 1
    $\begingroup$ yes it does ... it builds model at the child level that incorporate parent data e.g. hourly predictions based upon daily values or item predictions based upon a group value/prediction . At the end the child forecasts have to be reconciled in either a top-down or a bottoms up manner subject to a user selection. $\endgroup$ – IrishStat Apr 6 '18 at 19:44
  • $\begingroup$ OP posted this more than a year ago, we will see if he is still around ... $\endgroup$ – kjetil b halvorsen Apr 6 '18 at 19:48
  • $\begingroup$ I suggest that you download AUTOBOX and read the material on Parent to Child to get a full read of the approach to multi-level modelling. $\endgroup$ – IrishStat Apr 6 '18 at 20:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.