# Random Forest: IncNodePurity and Feature Selection for Binary Logistic Regression

After creating a Random Forest object using randomForest with around 500 candidate variables, I used importance(object) to display IncNodePurity for each of the candidate variables in relation to the binary outcome of interest (Payment/No Payment).

I am aware that IncNodePurity is the total decrease in node impurities, measured by the Gini Index from splitting on the variable, averaged over all trees. What I don't know is what should be the cutoff for candidate variables to be retained after making use of randomForest for feature selection in regards to binary logistic regression models. For example, the smallest IncNodePurity among my 498 variables is 0.03, whereas the largest IncNodePurity is 96.68. In summary, I have one main question:

Is there a cutoff for IncNodePurity? If yes, what is it?

If no, how do you determine the cutoff? Do you simply take 10 candidate variables with the largest IncNodePurity if you want a model with only 10 predictor variables?

Any thoughts or references are greatly appreciated. Thanks!

• This can be mean squared error obtained via cross-validation. Cross-validation is very useful trick, I'd suggest you to find its explanation in web - that would be more clear than explanations of mine. In R there is function cv.glm() in package boot that can help you. – O_Devinyak Sep 4 '13 at 18:23