2
$\begingroup$

So I know there is a feature_importances_ variable under the XGBoost classifier. I was wondering if there is a way to see the deciding features for each observation? This will allow me to understand why the machine learning algorithm predicted its class for each observation.

I looked into LIME: https://github.com/marcotcr/lime

And it looks to have what I need. I'm just curious if there are any other libraries that do something similar.

$\endgroup$
2
  • $\begingroup$ Check out medium.com/@ModelOriented/… for everything you might want to know (and more) about the state of the art of model explanation and interpretation in R. Przemyslaw and his group are really leaders in this regard. $\endgroup$
    – jbowman
    Feb 24, 2020 at 23:09
  • $\begingroup$ Sounds good, thanks. $\endgroup$
    – Jon
    Feb 25, 2020 at 19:39

2 Answers 2

1
$\begingroup$

Have a look at SHapley Additive exPlanations, which has a game theoretic basis ("How do you 'optimally' assign credit to different players in a team for the team outcome?" - here variables are treated like players). For xgboost and similar algorithms (e.g. LightGBM), the calculations - that involve considering all orders in which you could give features importance - can be done quite efficiently so that this approach has become quite popular. SHAP values have their flaws (like probably most other current explainability approaches): see e.g. this paper or the work discussed in this podcast episode.

More generally, there's a great online book on explainable ML that's worth looking into. Chapters 5.9 and 5.10 deal with SHAP, but there's also chapters on LIME etc., which are of course also well-known approaches.

$\endgroup$
0
$\begingroup$

You should have a look at this great tuto that gives interpretability insight for any model. In particular the package eli5 in Python and this kernel about permutation importance should help you to have a deeper understanding of which feature has a good predictive signal for a particular sample.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.