I am executing a lightGBM model to forecast my units sold (qty) over a period of time. Objective is to run a model for each product group and be able to capture the trends, price elasticity, etc and make relevant predictions in the future when future prices are known.
Features used (Xs):
- Mean price of product group (to incorporate price elasticity in the regression model, and account for how Y changes with changes in product group's price)
- time features (day of week, day of month, day of year, week, month, year)
- Lag of Qty sold (1 day lag, 2 day lag, 3 day lag, 2 day lag mean, 4 day lag mean)
- Campaign Calendar (one hot encoded campaigns based on their date. eg: Black Friday)
- Demand Qty
Earlier when I implemented the model without Lag of dependent variables included, the MAPE on test data is high (>30% average), however, as soon as I include the lag variables, the MAPE reduces to astounding and almost perfect (5-8%), which makes me think if my model overfits and I might not see the same accuracy on future data predictions.
My question is:
- Whether the above approach of adding lag variables makes sense? I know it is a good practice, but seeing such huge changes in MAPE makes me wonder if its the right way.
- If yes, how should I prepare my data to make future predictions? Should I execute the lightGBM regression one day at a time, looking at 3 days of lagged targets, and make predictions on predicted Y in a loop? I'm not clear with this so would appreciate any guidance on that