How do we make predictions for future data when you have lagged dependent features used in training?

Question

I am executing a lightGBM model to forecast my units sold (qty) over a period of time. Objective is to run a model for each product group and be able to capture the trends, price elasticity, etc and make relevant predictions in the future when future prices are known.

Features used (Xs):

• Mean price of product group (to incorporate price elasticity in the regression model, and account for how Y changes with changes in product group's price)
• time features (day of week, day of month, day of year, week, month, year)
• Lag of Qty sold (1 day lag, 2 day lag, 3 day lag, 2 day lag mean, 4 day lag mean)
• Campaign Calendar (one hot encoded campaigns based on their date. eg: Black Friday)

Target:

• Demand Qty

Earlier when I implemented the model without Lag of dependent variables included, the MAPE on test data is high (>30% average), however, as soon as I include the lag variables, the MAPE reduces to astounding and almost perfect (5-8%), which makes me think if my model overfits and I might not see the same accuracy on future data predictions.

My question is:

• Whether the above approach of adding lag variables makes sense? I know it is a good practice, but seeing such huge changes in MAPE makes me wonder if its the right way.
• If yes, how should I prepare my data to make future predictions? Should I execute the lightGBM regression one day at a time, looking at 3 days of lagged targets, and make predictions on predicted Y in a loop? I'm not clear with this so would appreciate any guidance on that

Answer 1

Assuming your MAPE is on a test set and not based on the fitted values:

Yeah using features that are your lagged target variable is pretty common when doing time series forecasting with LightGBM although it does require looping which is typically called a 'recursive' model. It typically gives you good results in the short term although could be significantly worse in the long term because you are feeding it's predictions back to it. If you are just fitting a single time series at a time you could look at a package I created: LazyProphet which handles the looping for you. Also uses piece wise linear basis functions to fit local trends which may be more stable in the long run than using AR lags.

If you are fitting your model to multiple series then you will have to forecast one step, then use that as your lag_1 and just keep appending to that for each step. There really is no super nice way to do it.

If you are looking at the MAPE of your fitted values then pivot instead to time series cross validation or using a holdout set, although you can overfit those too!