2
$\begingroup$

I am executing a lightGBM model to forecast my units sold (qty) over a period of time. Objective is to run a model for each product group and be able to capture the trends, price elasticity, etc and make relevant predictions in the future when future prices are known.

Features used (Xs):

  • Mean price of product group (to incorporate price elasticity in the regression model, and account for how Y changes with changes in product group's price)
  • time features (day of week, day of month, day of year, week, month, year)
  • Lag of Qty sold (1 day lag, 2 day lag, 3 day lag, 2 day lag mean, 4 day lag mean)
  • Campaign Calendar (one hot encoded campaigns based on their date. eg: Black Friday)

Target:

  • Demand Qty

Earlier when I implemented the model without Lag of dependent variables included, the MAPE on test data is high (>30% average), however, as soon as I include the lag variables, the MAPE reduces to astounding and almost perfect (5-8%), which makes me think if my model overfits and I might not see the same accuracy on future data predictions.

My question is:

  • Whether the above approach of adding lag variables makes sense? I know it is a good practice, but seeing such huge changes in MAPE makes me wonder if its the right way.
  • If yes, how should I prepare my data to make future predictions? Should I execute the lightGBM regression one day at a time, looking at 3 days of lagged targets, and make predictions on predicted Y in a loop? I'm not clear with this so would appreciate any guidance on that
$\endgroup$

1 Answer 1

2
$\begingroup$

Assuming your MAPE is on a test set and not based on the fitted values:

Yeah using features that are your lagged target variable is pretty common when doing time series forecasting with LightGBM although it does require looping which is typically called a 'recursive' model. It typically gives you good results in the short term although could be significantly worse in the long term because you are feeding it's predictions back to it. If you are just fitting a single time series at a time you could look at a package I created: LazyProphet which handles the looping for you. Also uses piece wise linear basis functions to fit local trends which may be more stable in the long run than using AR lags.

If you are fitting your model to multiple series then you will have to forecast one step, then use that as your lag_1 and just keep appending to that for each step. There really is no super nice way to do it.

If you are looking at the MAPE of your fitted values then pivot instead to time series cross validation or using a holdout set, although you can overfit those too!

$\endgroup$
8
  • $\begingroup$ Thanks for the detailed explanation! :) $\endgroup$ Sep 18, 2022 at 17:00
  • $\begingroup$ Honored to get an explanation from an author! Will try out the LazyProphet version and let you know if any questions. $\endgroup$ Sep 18, 2022 at 17:12
  • $\begingroup$ I have a question in implementing LazyProphet with exogenous features. If while fitting the model (using .fit) can I pass arguments as (x,y) instead of just (y)? In the documentation it was not mentioned so just wanted to confirm. $\endgroup$ Sep 19, 2022 at 10:25
  • $\begingroup$ hey thanks for trying it out. Yes I have a gist example I need to add to the documentation, you can find it here. Just make sure to use 2-d arrays for the exogenous data and feel free to open any issues you come across on github! $\endgroup$
    – Tylerr
    Sep 19, 2022 at 13:28
  • $\begingroup$ Thank you so much! I tried it and it worked! A few small clarifications: - In my test set, while making predictions, is LazyProphet using the actual lag values or making predictions first and using them as lags? Just want to ensure that when I make predictions on future data, the model should respond the same way it did for the test set. $\endgroup$ Sep 19, 2022 at 17:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.