Random forest regression produce different importance ranking

Question

I've been working a random forest regression model with my own datasets using the randomForest package.

The model produces a % var explained of ~27%. I also used the importance() function to produce a ranking of the relative importance of the variables, but everytime I use a different number for the set.seed() function, the random forest model produces a different ranking. It should be mentioned that I have several variables whose importance are relatively close.

I am wondering is there a consistent way to produce the ranking of the variable importance?

Answer 1

No- no consistent variable importance value is possible in this case.

The variable importance changes because the underlying model itself changes each time you change the seed value. Different models will have different values for variable importance.

One thing that you could do would be run many (say, 10) Random Forest models with different seed values and average the variable importance scores across the models- this would get you a better approximation of what you could expect variable importance to be on average from each individual Random Forest model you train on the dataset.

Answer 2

It should be noted that RFs are an approximating approach to modelling (typically) massive amounts of information, based on a large number of bootstrapped, finite data "mini-models." Only in the theoretical, infinite limit do they approach stable, consistent results. As such, and even with massive numbers of bootstrapped subsamples, they remain approximating estimators that are subject to change as a function of the random draws from the underlying empirical information. This result generalizes to all finite data resampling tools including the bootstrap, jacknife, data augmentation, RFs, k-fold CV, you name it.

In other words, there is only one way to reproduce the rankings from one RF to the next and that is by using the same seed. You can evaluate the extent to which the rankings are consistent using a Spearman correlation for ordinal relationships based on the different importance rankings

Answer 3

I have another opinion to the different importance ranking, there are following reasons for this variation:

First, maybe there some highly correlated variables. random forest can't remove the highly correlated variables because of independent select features and tree construct process, so the highly correlated variables importance ranking will change every time. You can remove some highly correlated variables.

Second, the parameters of random forest is not tuned. you can tuning the parameters, and plot the parameters vs. rmse, then select a more suitable parameters.

Using the corresponding method, maybe you can produce a more stable results.