After creating a Random Forest object using
randomForest with around 500 candidate variables, I used
importance(object) to display IncNodePurity for each of the candidate variables in relation to the binary outcome of interest (Payment/No Payment).
I am aware that IncNodePurity is the total decrease in node impurities, measured by the Gini Index from splitting on the variable, averaged over all trees. What I don't know is what should be the cutoff for candidate variables to be retained after making use of
randomForest for feature selection in regards to binary logistic regression models. For example, the smallest IncNodePurity among my 498 variables is 0.03, whereas the largest IncNodePurity is 96.68.
In summary, I have one main question:
Is there a cutoff for IncNodePurity? If yes, what is it?
If no, how do you determine the cutoff? Do you simply take 10 candidate variables with the largest IncNodePurity if you want a model with only 10 predictor variables?
Any thoughts or references are greatly appreciated. Thanks!