I have historic sales data from a bakery (daily, over 3 years). Now I want to build a model to predict future sales (using features like weekday, weather variables, etc.).
How should I split the dataset for fitting and evaluating the models?
- Does it need to be a chronological train/validation/test split?
- Would I then do hyperparameter tuning with the train and validation set?
- Is (nested) cross validation a bad strategy for a time-series problem?
Here are some links I came across after following the URL suggested by @ene100:
- Rob Hyndman describing "rolling forecasting origin" in theory and in practice (with R code)
- other terms for rolling forecasting origin are "walk forward optimization" (here or here), "rolling horizon" or "moving origin"
- it seems that these techniques won’t be integrated into scikit-learn in the near future, because “the demand for and seminality of these techniques is unclear” (stated here).
And this is another suggestion for time-series cross validation.