I am building a random forest in R and was wondering how to extract the most important variables. I am using a random forest to classify if a click is fraud or not, and the goal is to identify characteristics that increase the probability of a click being fraud. Would the importance() and varImpPlot() R functions be helpful in identifying these variables or are there any other ways? The idea is to describe to a nontechnical client what the most important variables are. Are there any plots I could use to show the importance of these variables?
3 Answers
Would the importance() and varImpPlot() R functions be helpful in identifying these variables or are there any other ways?
Yes. If you would like to stick to random forest algorithm, I would highly recommend using conditional random forest in case of variable selection / ranking. You can use the party package for fitting conditional random forest. In this algorithm the permutation-importance measure
gets corrected to account for spurious correlation by estimating conditional permutation-importance
metric for which $X_j$ is considered together with other strongly correlated variables with $X_j$.
For more reference please see Party publication.
This paper also indicates some good practices on using the varimp()
function:
- if all predictor variables are of the same type, use either
randomForest
orcforest
(...) asrandomForest
is computionally faster - if the predictor variables are of different types, use
party::cforst
with the default optioncontrols = cforest_unbiased
and premutation importancevarimp(obj)
- if the predictor variables are correlated, depending on your research question, conditional importance, vailable via
varimp(obj, conditional = TRUE)
forparty::cforest
can add to the understanding of your data
The idea is to describe to a nontechnical client what the most important variables are. Are there any plots I could use to show the importance of these variables?
The most basic way to present variable importance is to create horizontal bart plot with normalized importance score (summing up to 100).
If you trained a model with the randomforest
package, you can access the feature importance for your fit model with the importance
function.
Say you're fit model is named fit
, then you could visualize it as follows:
Via the built in method: varImpPlot(fit)
Via ggplot2:
# make dataframe from importance() output
feat_imp_df <- importance(fit) %>%
data.frame() %>%
mutate(feature = row.names(.))
# plot dataframe
ggplot(feat_imp_df, aes(x = reorder(feature, MeanDecreaseGini),
y = MeanDecreaseGini)) +
geom_bar(stat='identity') +
coord_flip() +
theme_classic() +
labs(
x = "Feature",
y = "Importance",
title = "Feature Importance: <Model>"
)
Yes!
Alternatively you can use the function vimp in the randomForestSRC package.
Or the varimp function in the cforest package.
You can just simply make a barplot with the variable importance values to display the strength of the importance of the variables. varImpPlot makes this automatically.