I'm having some difficulty understanding how to interpret variable importance output from the Random Forest package. Mean decrease in accuracy is usually described as "the decrease in model accuracy from permuting the values in each feature".
Is this a statement about the feature as a whole or about specific values within the feature? In either case, is the Mean Decrease in Accuracy the number or proportion of observations that are incorrectly classified by removing the feature (or values from the feature) in question from the model?
Say we have the following model:
require(randomForest) data(iris) set.seed(1) dat <- iris dat$Species <- factor(ifelse(dat$Species=='virginica','virginica','other')) model.rf <- randomForest(Species~., dat, ntree=25, importance=TRUE, nodesize=5) model.rf varImpPlot(model.rf)
Call: randomForest(formula = Species ~ ., data = dat, ntree = 25, proximity = TRUE, importance = TRUE, nodesize = 5) Type of random forest: classification Number of trees: 25 No. of variables tried at each split: 2 OOB estimate of error rate: 3.33% Confusion matrix: other virginica class.error other 97 3 0.03 virginica 2 48 0.04
In this model, the OOB rate is rather low (around 5%). Yet, the Mean Decrease in Accuracy for the predictor (Petal.Length) with the highest value in this measure is only around 8.
Does this mean that removing Petal.Length from the model would only result in an additional misclassification of 8 or so observations on average?
How could the Mean Decrease in Accuracy for Petal.Length be so low, given that it's the highest in this measure, and thus the other variables have even lower values on this measure?