2
$\begingroup$

I'm currently working on a large-scale time series framework at my job and encountering some frustration when it comes to XGBoost/Random Forest. The goal of this effort is to be able to forecast 18 months in the future, therefore necessitating the addition of recursive forecasting, as I will be doing more than a point forecast.

Unfortunately, even with Optuna, I'm struggling quite a bit to produce long horizon forecasts that are good, even plausible, as the data becomes more and more noisy as you traverse the horizon. I wanted to use this forum as a place to discuss some potential improvements that I could add. I'm tuning the amount of lags to use for autoregression, and also only using the target quantity as the response, no other exogenous or endogenous regressors. Is this something where unfortunately recursive forecasting is necessary or is there a different approach?

I should also include that this framework is responsible for forecasting 20,000 different items, and typical workarounds must be discarded in consideration of computational safety.

Improve this question
$\endgroup$
10
  • $\begingroup$ You can also tune models directly for long-term forecasts, in the extreme case having one model for 1 month ahead, a different model for 2 months ahead, and so forth. Long-range forecasting is inherently harder. Prediction intervals rather than point forecasts often make more sense. That said, this may be helpful. $\endgroup$
    – Stephan Kolassa
    Commented Jul 23 at 14:45
  • $\begingroup$ Thank you for the response @StephanKolassa ! I should've included that framework is responsible for over 20,000 different datasets, so training 18 models on each item isn't feasible (I have 10 other frameworks apart from XGBoost and Random Forest). It's frustrating because the latter 2 mentioned perform super well on the testing phase, but produce almost nonsensical forecasts. $\endgroup$
    – foodman561
    Commented Jul 23 at 14:48
  • $\begingroup$ What is a "dataset" for you? A single time series? $\endgroup$
    – Stephan Kolassa
    Commented Jul 23 at 14:50
  • $\begingroup$ Yes. Demand quantities and demand months, for a single item. Scale that by 20,000 $\endgroup$
    – foodman561
    Commented Jul 23 at 14:51
  • $\begingroup$ Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. $\endgroup$
    – Community Bot
    Commented Jul 23 at 14:53

1 Answer 1

Reset to default
2
$\begingroup$

Have you had a look at how people approach this on Kaggle? There's plenty of examples of having to predict sales for multiple timepoints. I don't think recursive approach (i.e. where you predict one timepoint and then use that prediction as an input to the prediction for the next timepoint) is a common way of doing this. Alternatives include:

  • Train tree-based model using data up to time x as the input features, but as additional features which day you are predicting (including how far it is in the future, what week day, month etc.). You just end up creating more records from your training data (e.g. predict 10 July 2023 once based on the data incl. 9 July 2023, once based on the data 1 month before that etc., in whatever timesteps make sense). Doing that should even let you get your prediction uncertainty roughly right (predicting further out should usually harder), if you do something like conformal prediction.
  • Train a model that can output predictions for multiple future times at the same time (e.g. suitable neural networks can do that, but there's many other things, just be aware stuff like Facebook's Prophet have developed a bit of a bad reputation for too often getting beaten by ARIMA).

Also, don't forget to benchmark against something very simple and evaluate properly (e.g. past-vs-future validation set-up) to see whether your complex model is any good.

Improve this answer
$\endgroup$
1
  • $\begingroup$ yes I'm using both NN's and Prophet (Prophet stinks though). these work well. also benchmarking. im specifically pointing to the issue arising in building a 'from-scratch' XGB and Random Forest model, as you have to reformat the data to be a supervised learning problem $\endgroup$
    – foodman561
    Commented Jul 23 at 14:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.