6
$\begingroup$

related to Feature importance for random forest classification of a sample

This blog by Ando Saabas suggests a nice way to interpret a tree result for a specific sample into per-feature contributions. Basically he goes over the path to the sample and counts each delta in the prediction $pred(currentNode) - pred(parentNode)$ as a contribution for the splitting feature.

Can I use a similar approach for interpreting results of gradient boosting with tree weak classifiers, specifically for the popular xgboost implementation?

Improve this question
$\endgroup$
2
  • $\begingroup$ This approach doesn't make a whole lot of sense because it doesn't account for the weights of the leaf nodes. Node weights are the "boosting" portion of the model, so ignoring them implicitly ignores the fact that you're using boosted trees. $\endgroup$
    – Sycorax ♦
    Commented Jul 3, 2018 at 14:17
  • $\begingroup$ this is also implemented in eli5, which includes xgboost/lightgbm/sklearn/'catboost`. github.com/eli5-org/eli5/blob/master/eli5/lightgbm.py $\endgroup$
    – Joey Gao
    Commented Mar 19, 2021 at 13:26

1 Answer 1

Reset to default
3
$\begingroup$

since posting this, I learned that xgboost does make use of this method exactly, available when calling predict(pred_contribs=True, approx_contribs=True).

Improve this answer
$\endgroup$
2

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.