# Random forest variable importance: Mean minimal depth and number of nodes disagree

I'm trying to determine variable importance for a random forest with 8 predictors, and different variable importance measures are telling different stories. The forest was generated in R with randomForest (with localImp=TRUE), and the measures come from randomForestExplainer's measure_importance (with the default mean_sample="top_trees").

> fstLH <- randomForest(Blackness ~ .,
resultsWithProsod[resultsWithProsod$Accent=="LH", !(colnames(resultsWithProsod) %in% ignoreCols)], localImp=TRUE) > impFrmLH <- measure_importance(fstLH) > c(lapply(impFrmLH[2], function(x) { setNames(sort(x, decreasing=F), impFrmLH$variable[order(x, decreasing=F)])
}),
lapply(impFrmLH[-(1:2)], function(x) {
setNames(sort(x, decreasing=T), impFrmLH$variable[order(x, decreasing=T)]) }))$mean_min_depth
PhraseSpeechRate        PeakDelay              HNR     IntensityAvg
2.003904         2.007264         2.116000         2.119136
F0MaxMinRatio          Shimmer           Jitter             Step
2.185136         2.355136         2.366000         3.093680

$no_of_nodes F0MaxMinRatio IntensityAvg HNR Jitter 2866 2837 2827 2762 Shimmer Step PhraseSpeechRate PeakDelay 2684 2198 1486 1453$mse_increase
PhraseSpeechRate        PeakDelay              HNR     IntensityAvg
0.091133749      0.080856088      0.048558841      0.048251741
F0MaxMinRatio          Shimmer           Jitter             Step
0.038783659      0.018955181      0.015282364      0.004449853

$node_purity_increase PhraseSpeechRate PeakDelay IntensityAvg HNR 97.10995 79.07907 54.43092 49.60133 F0MaxMinRatio Shimmer Jitter Step 48.69803 30.74844 28.62819 11.86899$no_of_trees
HNR           Jitter    F0MaxMinRatio     IntensityAvg
500              500              499              499
Shimmer             Step PhraseSpeechRate        PeakDelay
499              495              486              476

$times_a_root PeakDelay PhraseSpeechRate HNR F0MaxMinRatio 114 103 79 57 Jitter Shimmer IntensityAvg Step 54 47 46 0$p_value
PeakDelay PhraseSpeechRate             Step          Shimmer
1.000000e+00     1.000000e+00     9.999887e-01     1.198402e-10
Jitter              HNR     IntensityAvg    F0MaxMinRatio
7.488331e-16     4.964456e-21     6.792990e-22     1.685192e-24


According to mean minimal depth, MSE increase, node purity increase, and times-a-root, two predictors (PhraseSpeechRate and PeakDelay) are clearly most important to the forest. But these two predictors also show up in the fewest nodes and fewest trees, and they get a (suspicious) p-value of 1.

Is there some interaction I'm missing here, or do I need to adjust tuning parameters of the random forest?

• They all measure different things, so there's no underlying reason they would agree. – Matthew Drury Aug 3 '18 at 2:56
• Right, I'm not saying they should all be in lockstep, but certain disagreements are curious. Intuitively, I would expect that if a variable is likelier than any other to be a root node, then it would also likely show up frequently as a non-root node; but PhraseSpeechRate and PeakDelay are the commonest root nodes but least common (root or non-root) nodes. – Dan Villarreal Aug 3 '18 at 3:50
• Perhaps a better way to frame the question would've been this: under what conditions would we expect these variable importance measures to (strongly) disagree in this way? – Dan Villarreal Aug 3 '18 at 3:58
• It's hard to tell without having the complete code. However, the p-value does not indicate which variables are most important. An informative variable would probably split near the root node, so would be less likely to split further down (still possible) but have small minimal depth. You can see in the following vignette that sometimes variables with relatively smaller minimal depth have larger p-values: cran.rstudio.com/web/packages/randomForestExplainer/vignettes/… – Peter Calhoun Aug 5 '18 at 6:36
• I see. I thought the high rate of selection as a root node might trigger a lower rate of splitting further down the tree. And I've added the rest of the code. – Dan Villarreal Aug 5 '18 at 23:05