Aside from ordinary tree boosting, XGBoost offers DART and gblinear. On DART, there is some literature as well as an explanation in the documentation. However, I can't find any useful information about how the gblinear booster works. I would like to know which exact model is used as base learner, and how the algorithm is different from the normal tree boosting algorithm. Does anybody here know this?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
It is just using a linear model with l1 and l2 regularization as its base learner rather than a decision tree. Here is a similar Q&A: Difference in regression coefficients of sklearn's LinearRegression and XGBRegressor .
So it will be different than other linear models because it is optimized slightly differently but more-so you are boosting it which provides further regularization in linear models unlike when you boost trees and add complexity. So it tends to shrink the linear coefficients. You can boost any model but you typically only get major gains when you boost models which partition your data in some way such as linear piecewise functions or trees.
