7
$\begingroup$

Permutation feature importance (PFI) is a nice way of getting feature importance in black-box models or models where it is difficult to characterise the relationship between the features and the response. However, it suffers from when the features are highly correlated, which can lead to weird results.

Anyone has experienced with this technique on a correlated dataset? Two solutions I am thinking about are:

  1. Remove the highly colinear altogether.
  2. Group the highly correlated together and shuffle their values at the same time (rather than doing it one column at a time) to get a single importance score for that group of correlated features.

Anyone has other propositions?

Improve this question
$\endgroup$
2
  • $\begingroup$ In the spirit of RF you can do bagging, which should remove the correlations. $\endgroup$
    – user2974951
    Sep 27, 2018 at 6:33
  • $\begingroup$ It might help to clarify what the "weird results" are that you see when doing PFI "the normal way" on datasets with correlated features, and which you're trying to avoid by the altered method. $\endgroup$
    – R.M.
    Oct 1, 2018 at 20:48

1 Answer 1

Reset to default
2
$\begingroup$

Regularize. If you regularize your model and inputs, you should remove the issue of multi-collinearity.

Correlation is not causation, however, and so while you can use your method to assign "predictive power" to your features, you cannot establish any sort of causal relationship.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.