1
$\begingroup$

I have a dataset of 14558 rows and 250 variables. I am trying to solve a classification problem thanks to r party package and the cforest function (which corresponds to a Random Forest).

I would like to obtain the value of variables importance for each tree instead of only getting the mean of the variable importance over all trees (this is what the the varimp function does).

I have in mind to do a boxplot with those variables importance value in order to evaluate the confidence of the value I have. If someone has another idea to do it, please let me know ! :)

I was thinking that it would be possible since we see it in the following paper : https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-9-307 Importance variables

I have search over this forum and the party package documentation and did not find the solution. I hope I am not opening an already existing topic !

Thanks for your help :)

Rémi

$\endgroup$

closed as off-topic by StatsStudent, Michael Chernick, Dimitris Rizopoulos, Gavin Simpson, Ferdi Jan 9 at 6:00

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question appears to be off-topic because EITHER it is not about statistics, machine learning, data analysis, data mining, or data visualization, OR it focuses on programming, debugging, or performing routine operations within a statistical computing platform. If the latter, you could try the support links we maintain." – StatsStudent, Michael Chernick, Dimitris Rizopoulos, Gavin Simpson, Ferdi
If this question can be reworded to fit the rules in the help center, please edit the question.

2
$\begingroup$

To obtain a measure of uncertainty for the permutation-based variable importance, you can simply generate a large number of random permutations and then look at their distribution. This is what we have done in the figure from the manuscript you cite.

Note that this is quite different from looking at the variable importances of the individual trees which would likely be misleading. The individual trees can be quite weak classifiers and hence have misleading importances. So you should really assess the variability of the forest importances not the tree importances.

The varimp() implementation in the party package by default just uses a single random permutation of each variable. But it allows to set varimp(..., nperm = ...) to a larger number. However, in that case it only returns the mean of the nperm permutations. But if you want the full distribution you can also easily replicate(..., varimp(...)) the desired number of times.

For example, you can fit the forest from the ?varimp examples:

library("party")
set.seed(1)
cf <- cforest(score ~ ., data = readingSkills,
  control = cforest_unbiased(mtry = 2, ntree = 50))

And for each variable you can generate, say, 50 random permutations for each predictor and visualize these with a boxplot:

vi <- t(replicate(50, varimp(cf)))
boxplot(vi)

varimp

$\endgroup$
  • $\begingroup$ Thank you very much, this is very instructive. I was a bit doubtful about taking the vimp of each tree and your explanation enlightens me and confirms my doubts ! Have a nice day, Rémi $\endgroup$ – Rémi Boutin Jan 9 at 9:17
  • $\begingroup$ You're welcome. If the answer was useful to you, please accept it through the system (if you can still do that, given that it is "on hold"). And in case someone else reads this: I have answered the question here (rather than moving it to SO) because the crucial point was the conceptual understanding of variable importances in forests vs. trees. $\endgroup$ – Achim Zeileis Jan 9 at 13:43

Not the answer you're looking for? Browse other questions tagged or ask your own question.