I'm using the randomForest R package to perform a random forest feature selection. I undestand that, after the execution of the randomForest function, I have to check the importance field, and study the importance measured throudh mean square error accuracy reduction and Gini purity reduction.
For example, by using this R code:
data(iris) library("randomForest") set.seed(71) iris.rf <- randomForest(Species ~ ., data=iris, importance=TRUE, proximity=TRUE) print(iris.rf)
It will print out:
MeanDecreaseAccuracy MeanDecreaseGini Sepal.Width 0.007962441 2.625413 Sepal.Length 0.031901722 10.714741 Petal.Length 0.304760304 42.104241 Petal.Width 0.300907912 43.767952
Then, I see that I can rank these features by the MeanDecreaseAccuracy or MeanDecreaseGini field to understand what are the most important ones. Even if I understand the output of the method, I cannot understand how the method obtains its results.
The questions are:
- How does the method compute the accuracy?
- How does the method split the dataset into training set and test set to compute the accuracy?