2
$\begingroup$

When you train your XGBoost regression model, you can obtain feature importances by using:

model.get_score(importance_type="gain")

Although I tried to reconstruct the value and have done some research on it, I am still struggling to figure out, how gain is computed in XGBoost?

It is partially explained here: Relative variable importance for Boosting

But this is focused on classification. And even though there is a link saying that squared error with Friedman's improvement score is used, I have not reached the same numbers.

Improve this question
$\endgroup$
1
  • $\begingroup$ I need to edit that answer of mine. In fact, regression trees are always used in boosting. So it is subtly incorrect. $\endgroup$ – Matthew Drury Sep 7 '17 at 14:42
2
$\begingroup$

After all it seems that gain is computed in quite complicated manner and it's explanation can be found at https://xgboost.readthedocs.io/en/latest/tutorials/model.html at the bottom of the page.

Gain is a metric defined by XGBoost and it also involves evaluation of the structure of the tree.

Due to the complexity of the explanation, it is not copied in here.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.