2
$\begingroup$

I have a black box random forest model where I can only feed some input data in matrix form and receive the predictions in a vector. Is it possible to obtain any information about the feature importances by just feeding it data and looking at how the predictions change? I also have access to the data the model was trained on, but have no access to any other information of the model itself.

I've previously done this with a black box linear regression model by feeding it identity matrices, but does that not work in the case of a random forest?

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ That won't work for random forests or for many other models. I don't think it would work for any model that accounts for interactions between the input features because, in such models, you need to change two variables at a time to see the effect. In addition, with something like a random forest, there are threshold effects to account for. $\endgroup$
    – roundsquare
    Oct 5, 2019 at 22:43

1 Answer 1

Reset to default
2
$\begingroup$

A model agnostic way to assess variable importance is permutation importance. Model agnostic means it works for any type of model.

How is it calculated?

1) Choose evaluation metric and calculate original performance on your query data, e.g. an independent validation set.

2) Select input x. Shuffle column x in the data and (without refitting) get predictions. Calculate performance on shuffled data. Variable importance of x is assessed by calculating difference (or relative change) in performance before and after shuffling.

3) Repeat 2) for all covariables of interest.

You can use R packages DALEX, iml or flashlight (mine) to calculate such measure.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.