Can we use any learners in gradient boosting instead of trees?

Question

As we are simply trying to predict residuals from weak learners and aggregating them, can we use any weak learners in gradient boosting machines instead of trees ? If so, why are the all the gbm implementations like xgboost, lightgbm use trees ?

Answer 1

Yes, you can use learners besides trees. XGBoost implements linear learners in addition to trees.

Trees are nice because weak learners can be aggregated to become strong learners. We have several posts about this.

• In boosting, why are the learners "weak"?
• On the "strength" of weak learners