With some research I found that it updates the leaves (does not create new or remove old ones) is it right? How this happens?

Another question is when the incremental learning is done in concept shifting data, is LightGBM good to deal with this problem?


1 Answer 1

  1. LightGBM will add more trees if we update it through continued training (e.g. through BoosterUpdateOneIter). Assuming we use refit we will be using existing tree structures to update the output of the leaves based on the new data. It is faster than re-training from scratch since we do not have to re-discover the optimal tree structures. Nevertheless, please note that, almost certainly, it will have worse performance (on the combined old and new data) than doing a full retrain from scratch on them.
  2. Any online learning algorithm will be designed to adapt to changes. That said, LightGBM's performance will depend on the training parameters we will use and how we will validate our predictions (e.g. how much we care to disregard previous data points). Assuming we properly train our booster without having a relevant baseline (e.g. a ridge regression trained incrementally), it does not make sense to say "LightGBM is good (or bad)" for dealing with concept drift.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.