Long horizon recursive forecasting using XGBoost/Random Forest

Question

I'm currently working on a large-scale time series framework at my job and encountering some frustration when it comes to XGBoost/Random Forest. The goal of this effort is to be able to forecast 18 months in the future, therefore necessitating the addition of recursive forecasting, as I will be doing more than a point forecast.

Unfortunately, even with Optuna, I'm struggling quite a bit to produce long horizon forecasts that are good, even plausible, as the data becomes more and more noisy as you traverse the horizon. I wanted to use this forum as a place to discuss some potential improvements that I could add. I'm tuning the amount of lags to use for autoregression, and also only using the target quantity as the response, no other exogenous or endogenous regressors. Is this something where unfortunately recursive forecasting is necessary or is there a different approach?

I should also include that this framework is responsible for forecasting 20,000 different items, and typical workarounds must be discarded in consideration of computational safety.

Answer 1

Have you had a look at how people approach this on Kaggle? There's plenty of examples of having to predict sales for multiple timepoints. I don't think recursive approach (i.e. where you predict one timepoint and then use that prediction as an input to the prediction for the next timepoint) is a common way of doing this. Alternatives include:

• Train tree-based model using data up to time x as the input features, but as additional features which day you are predicting (including how far it is in the future, what week day, month etc.). You just end up creating more records from your training data (e.g. predict 10 July 2023 once based on the data incl. 9 July 2023, once based on the data 1 month before that etc., in whatever timesteps make sense). Doing that should even let you get your prediction uncertainty roughly right (predicting further out should usually harder), if you do something like conformal prediction.
• Train a model that can output predictions for multiple future times at the same time (e.g. suitable neural networks can do that, but there's many other things, just be aware stuff like Facebook's Prophet have developed a bit of a bad reputation for too often getting beaten by ARIMA).

Also, don't forget to benchmark against something very simple and evaluate properly (e.g. past-vs-future validation set-up) to see whether your complex model is any good.