6
$\begingroup$

In xgboost (xgbtree), gamma is the tunning parameter to control the regularization. I understand what regularization means in xgblinear and logistic regression, but in the context of tree-based methods, I'm not sure how regularization works.

Can someone explain how regularization works in xgbtree?

$\endgroup$
3
$\begingroup$

In tree-based methods regularization is usually understood as defining a minimum gain so which another split happens:

Minimum loss reduction required to make a further partition on a leaf node of the tree. The larger gamma is, the more conservative the algorithm will be.

Source: https://xgboost.readthedocs.io/en/latest/parameter.html

This minimum gain can usually be set for anything between $(0,\infty)$.

Here's a somewhat good article on how to tune regularization on XGBoost.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.