Online Tree Based Algorithms

Question

Linear regression and logistic regression can do online training(i.e. continuous training as new data arrives) via stochastic gradient descent. Are there any tree based algorithms which can efficiently do online training? The problem with linear regression and logistic regression is both create only linear models and are poor at handling non-linear data.

Answer 1

I had similar a question and initially stumbled upon your question while researching. I decided to open a new question and now want to share my insights - maybe it proofs helpful to future visitors (like me back then):

The paper Learning in Nonstationary Environments: A Survey and the respective Wikipedia article regarding Incremental Decision Trees provide an overview.

Some algorithms proposed (non-exhaustive selection; see provided paper/article above):

• Very Fast Decision Trees (VFDT)
• CVFDT to adapt to concept drift and several modifications [...]
• Extremely Fast Decision Tree
• Hoeffding Trees
• (Learn++.NSE)
• (ONSBoost)

Some of those algorithms come with research paper implementations on github or are implemented as part of the scikit-multiflow library or the more recent River library.

Hope this helps and provides a sufficient basis for deeper research.