How to calculate variable importance taking correlated predictors into account?

This is the problem I am facing right now:

I have a dataset with 100.000 samples and 20 predictors. The predictors are correlated with each other due to their nature. I've run two different random forest models with a sample of the dataset (see why I used a subsample in the explanation below):

1. The first random forest was from the party package
2. The second random forest was from the randomForest package

My objective, aside from predicting a certain variable with the model, is to find out the importance of the predictors.

According to the test I ran with a subsample of the entire dataset, I got contraddictory answers on variable importance using party and randomForest.

Party, when using the function:

varimp(cf_fit,conditional=TRUE)

with the argument conditional set to TRUE, seems to output an answer that "makes more sense" (based on my knowledge of the subject) compared to the answer of the randomForest's

importance()

function.

By reading a bit deeper into the literature, I found out that this is common when using correlated predictors. Since the party package is partially doing a better job than the alternatives, I would like to use it, however, when feeding the whole dataset, the function varimp(cf_fit,conditional=TRUE) crashes the program. Again, I read that this is quite common or, at least, it is common for this function to be really slow, which it definitely is, even when using only a subsample.

Finally, my question:

How can I calculate the variable importance accounting for variable correlation? (ie: how can I do what the function varimp(cf_fit,conditional=TRUE) is doing so slowly at best, and crashing at worst?)

1) I simply don't do it this way and try to find the best predictors by running different times the random forest with different subset of predictors (possibly those with the least correlation among themselves), then I choose the independent predictors yielding the best results and only then I calculate variable importance.

2) Use a faster machine and try using party's varimp(cf_fit,conditional=TRUE) function anyway. I'm not sure this is a hardware problem... speculation time... is it maybe related to the way this function works?

Thanks to everyone in advance!

• What do you seek to understand by getting 'variable importance'? What do you plan to use the information for? It's a pretty nebulous and poorly defined idea. Here's a good discussion on the topic. – Dex Groves Nov 3 '16 at 17:15
• @DexGroves Hi, thanks for the link. I think I understand what you mean. My aim is to find the best predictors for the dependent variable I'm trying to predict, then run the random forest with that subset and compare the results. However, I'm required to produce the "variable importance metric" that can be obtained from the random forest and getting two very different results is a little bit worrying. – mickkk Nov 3 '16 at 17:47