1
$\begingroup$

I am wondering if there is a relationship between the performance quality of a predictive model (e.g., a classifier) and the reliability of corresponding feature importance values?

To give an example: I created two binary classifiers using two different sets of features. One classifier performs slightly better than the other. However, both perform not that great (balanced accuracies are around 65% and 55%). I calculated feature importance scores using permutation importance. Is it valid to say that both lists of importance values are equally reliable/interpretable?

Any help or literature recommendations are highly appreciated!

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

First of all permutation importance does not take into account some kinds of dependencies. For example if feature 1 and 2 are only important when they are both present, then with permutation importance you will never see this.

Besides for that - it depends by what you mean when you say feature importance. If you mean how important a feature is for a model, then it does not matter how well performing the model is. If you mean how strong of a connection does the feature have with the target, then yes a model's feature importance score is less relevant for less well performing model.

Besides that- you want help or recommendations in what?

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks, this is already helpful. $\endgroup$
    – RamsesII
    Commented Nov 17, 2022 at 13:58
  • $\begingroup$ Can you elaborate a bit more on your second paragraph? Where exactly is the difference? I assume this means that with permutation importance, the first case applies and it does not matter how well performing the model ist, right? Whereas for regression coefficients, for instance, which are a direct measure of the connection between feature and target, model performance has an impact? $\endgroup$
    – RamsesII
    Commented Nov 17, 2022 at 14:03
  • $\begingroup$ yes I think you understood me. I mean are you looking for (a) model prediction explanations or (b) which features are most important for predicting some target variable? In the case of (a) the model accuracy does not matter. All that matters are which features affect the model's decision making (for example LIME and SHAP do not take into account model accuracy). If you care about about (b) then model accuracy is very relevant. $\endgroup$
    – user3494047
    Commented Nov 17, 2022 at 14:25
  • $\begingroup$ Thanks for clarifying. I think in my case both (a) and (b) are important (or I don't get the difference after all)... like when I see that feature X has the highest permutation importance, I assume that it is the most important feature for predicting the outcome/target since permuting it decreases model performance a lot. Right? $\endgroup$
    – RamsesII
    Commented Nov 18, 2022 at 18:17
1
$\begingroup$

Yes feature importance is computed based on the impact on performance of the model. A feature with high importance means it has large impact on the performance for example, from 90% it came down to 70%. And when the performance measures are itself on the lower side, impact on performance does not make much sense. In a way you could relate performance and feature importance.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.