2
$\begingroup$

enter image description here

I am not able to understand what this plot depicts and what we mean by the relation between rankings.Let me add a few more things to make it a bit precise. This plot is generated by using plot_importance_rankings function available as part of randomForestExplainer package.

A sample code which generates this code is as

plot_importance_rankings(var_importance_frame)

where var_importance_frame contains important variables which we get as from

var_importance_frame <- measure_importance(rf_model)

Here rf_model is the trained random forest model. A sample example can be found at this link

$\endgroup$
  • 1
    $\begingroup$ Welcome to SE.CV; fun little question there! (+1) I provided an answer below, let me know if you need further clarifications. $\endgroup$ – usεr11852 Feb 7 at 2:23
0
$\begingroup$

What this plots shows is a pairwise comparison of the variable importance rank associated with each metric. In particular:

  • The lower diagonal part of the lattice shows the scatter plot between the variable importance ranks induced each of the two metrics on each variable. The blue line shows a LOESS smoother estimate and the grey bands the confidence intervals around it.
  • The diagonal of the lattice shows the density of the scatter plot (a relatively uninformative plot I would think personally). Effectively it tells us how the distribution of the ranks induced by the metric associated with that column looks like. (Notice that the lower right diagonal would be better rotated by 90$^o$ and then taken as a mirror image; it ends at around 10 as it effectively smooths along the $y$-axis of the lower row...)
  • The upper diagonal part shows the pairwise correlation between the variable importance ranks; that is effectively the Spearman correlation coefficients.

To demystify the plot a bit further let's focus on the upper most, lower diagonal sub-plot; cell (2,1) in matrix notation. That sub-plot shows how the metric mean_min_depth (i.e. the average minimum depth that a variable $x_j$ appears in the trees of a random forest) is associated with the metric accuracy_decrease (i.e. the mean decrease of prediction accuracy after a variable $x_j$ is permuted). Using these two metrics we can induce a ranking on the explanatory variables in our random forest. For example, which is the variable that when permuted causes the largest decrease in accuracy, the second largest decrease, etc. etc. This gives us the ranking of variables based on accuracy_decrease. Similarly we can have a ranking based on the metric mean_min_depth. Which variable has the smallest mean minimum depth (i.e. is usually used first when creating a tree), the second smallest mean minimum depth, etc. etc. Now this means that we have a two vectors with values between $1$ and $p$, $p$ being the total number of explanatory variables in our forest. The first vector was created by using the metric mean_min_depth and the second vector the metric accuracy_decrease. If we now do that scatter plot we will get the plot shown in left-most subplot of the second row (counting from the top). We can see here for example that this is almost linear. The first and the second ranking variables are the same; the third ranking variable from mean_min_depth is the fourth ranking variable from accuracy_decrease and vice versa, etc. This correlation is actually extremely strong. We can see that the value is $0.964$. We can therefore conclude that these two metrics effectively rank the variable in the same way. Let me note that these LOESS augmented scatter-plots come to their own when usually we have more than a few dozen variables and we can really see how the rankings are changing across many variables.

In conclusion, this composite plot shows the pairwise association of the variance importance ranks associate with different metrics. It highlights the fact that no single metric offers a unique and all-encompassing view. We can also directly see how choosing one metric over another would influence the ranking of our variable importances.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.