0
$\begingroup$

I have trained an XGBoost binary classifier and I would like to extract features importance for each observation I give to the model (I already have global features importance).

More specifically, I am looking for a way to determine, for each instance given to the model, which features have the most impact and make the input belong to one class or another. I would like to know something like the top 5 features which make the observation belong to some class and indications on how I should modify these 5 features so that the probability of belonging to this class decreases or increases.

For example, let’s say my model predicts whether a house costs more than 100,000 dollars (this is the positive class) based on its location, surface and number of bedrooms. I give it the following input: London, 400 square foots, 4 bedrooms and my model predicts a probability of 56% for the house to be in the positive class. I am looking for a Python module or a function that would show the most influential features for each observation.

$\endgroup$

closed as off-topic by mkt, Michael Chernick, Peter Flom Aug 3 at 11:29

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question appears to be off-topic because EITHER it is not about statistics, machine learning, data analysis, data mining, or data visualization, OR it focuses on programming, debugging, or performing routine operations within a statistical computing platform. If the latter, you could try the support links we maintain." – mkt, Michael Chernick, Peter Flom
If this question can be reworded to fit the rules in the help center, please edit the question.

2
$\begingroup$

Haven't got enough reputation to comment, so have to answer here. Have you tried SHAP values?

Rather easy to implement, and gives really intuitive answers. You can find a quick example with XGBoost here.

Alternatively, there's LIME, however I prefer SHAP myself.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.