After running a Random Forest Classifier on the Iris data set, I get an output that looks like this:

              setosa  versicolor virginica MeanDecreaseAccuracy MeanDecreaseGini
SLength     1.277324  1.632586   1.758101   1.2233029           9.173648
SWidth      1.007943  0.252736   1.014141   0.6293145           2.472105
PLength     3.685513  4.434083   4.133621   2.5139980          41.284869
PWidth      3.896375  4.421567   4.385642   2.5371353          46.323415

While I understand the last 2 columns, I don't understand what the numbers in the first 3 columns (the classes) are.

Can someone help me understand what the number 1.277324 (SLength, setosa) means please?

  • $\begingroup$ is that in R? If so please add tag or mention it in question. $\endgroup$ – Antoine Aug 19 '15 at 18:29

For multi-classification(classes >2), variables can vary in their usefulness to separate and predict given classes. Therefore a separate out-of-bag delta-%misclassifcation is computed during training for each class on C++ level code and later aggregated to the total variable importance. As a rule of thumb(+/- 5%): When training a class balanced RF model, the mean value of class specific variable importances is equal to the total variable importance. But classes unbalanced, implemented classweights, changed cutoff, low ntree will distort this thumb-rule.

So 1.277324 (SLength, setosa) means that permuting Sepal.length increased the misclassification error of OOB Setosa class samples by 1.27%.

All variable importances in IRIS are fairly low, because petal.width and petal.length are strong redundant predictors. Try remove either petal.xxx from data set and retrain.

The thumb rule can be verified here:

outImpApprox = sapply(1:10, function(thisRepeat){
  rf = randomForest(Species ~., data = iris,importance=T,ntree=1000) 
  mean.class.specific.imp = apply(rf$importance[,1:3],1,mean)
      total.imp = rf$importance[,4]
  relative.approx.deviation = mean.class.specific.imp/total.imp

From help file, value;importance:

a matrix with nclass + 2 (for classification) or two (for regression) columns. For classification, the first nclass columns are the class-specific measures computed as mean descrease in accuracy. The nclass + 1st column is the mean descrease in accuracy over all classes.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.