4
$\begingroup$

I am building a random forest in R and was wondering how to extract the most important variables. I am using a random forest to classify if a click is fraud or not, and the goal is to identify characteristics that increase the probability of a click being fraud. Would the importance() and varImpPlot() R functions be helpful in identifying these variables or are there any other ways? The idea is to describe to a nontechnical client what the most important variables are. Are there any plots I could use to show the importance of these variables?

$\endgroup$
1
  • 1
    $\begingroup$ Just in case you want to give it a try : The Boruta algorithm, provided by the Boruta package, is nice. $\endgroup$ Dec 3, 2015 at 14:51

3 Answers 3

2
$\begingroup$

Would the importance() and varImpPlot() R functions be helpful in identifying these variables or are there any other ways?

Yes. If you would like to stick to random forest algorithm, I would highly recommend using conditional random forest in case of variable selection / ranking. You can use the party package for fitting conditional random forest. In this algorithm the permutation-importance measure gets corrected to account for spurious correlation by estimating conditional permutation-importance metric for which $X_j$ is considered together with other strongly correlated variables with $X_j$.

For more reference please see Party publication.

This paper also indicates some good practices on using the varimp() function:

  • if all predictor variables are of the same type, use either randomForest or cforest (...) as randomForest is computionally faster
  • if the predictor variables are of different types, use party::cforst with the default option controls = cforest_unbiased and premutation importance varimp(obj)
  • if the predictor variables are correlated, depending on your research question, conditional importance, vailable via varimp(obj, conditional = TRUE) for party::cforest can add to the understanding of your data

The idea is to describe to a nontechnical client what the most important variables are. Are there any plots I could use to show the importance of these variables?

The most basic way to present variable importance is to create horizontal bart plot with normalized importance score (summing up to 100).

$\endgroup$
2
$\begingroup$

If you trained a model with the randomforest package, you can access the feature importance for your fit model with the importance function.

Say you're fit model is named fit, then you could visualize it as follows:

Via the built in method: varImpPlot(fit)

Via ggplot2:

  # make dataframe from importance() output
  feat_imp_df <- importance(fit) %>% 
    data.frame() %>% 
    mutate(feature = row.names(.)) 

  # plot dataframe
  ggplot(feat_imp_df, aes(x = reorder(feature, MeanDecreaseGini), 
                         y = MeanDecreaseGini)) +
    geom_bar(stat='identity') +
    coord_flip() +
    theme_classic() +
    labs(
      x     = "Feature",
      y     = "Importance",
      title = "Feature Importance: <Model>"
    )
$\endgroup$
0
$\begingroup$

Yes!

Alternatively you can use the function vimp in the randomForestSRC package.

Or the varimp function in the cforest package.

You can just simply make a barplot with the variable importance values to display the strength of the importance of the variables. varImpPlot makes this automatically.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.