I'm using the randomForest R package to perform a random forest feature selection. I undestand that, after the execution of the randomForest function, I have to check the importance field, and study the importance measured throudh mean square error accuracy reduction and Gini purity reduction.
For example, by using this R code:
data(iris)
library("randomForest")
set.seed(71)
iris.rf <- randomForest(Species ~ ., data=iris, importance=TRUE,
proximity=TRUE)
print(iris.rf)
It will print out:
MeanDecreaseAccuracy MeanDecreaseGini
Sepal.Width 0.007962441 2.625413
Sepal.Length 0.031901722 10.714741
Petal.Length 0.304760304 42.104241
Petal.Width 0.300907912 43.767952
Then, I see that I can rank these features by the MeanDecreaseAccuracy or MeanDecreaseGini field to understand what are the most important ones. Even if I understand the output of the method, I cannot understand how the method obtains its results.
The questions are:
- How does the method compute the accuracy?
- How does the method split the dataset into training set and test set to compute the accuracy?