9
$\begingroup$

I dont get the difference between the rfobject$importance and importance(rfobject) in the MeanDecreaseAccuracy column.

Example:

> data("iris")
> fit <- randomForest(Species~., data=iris, importance=TRUE)
> fit$importance
                  setosa  versicolor   virginica MeanDecreaseAccuracy MeanDecreaseGini
Sepal.Length 0.027078501 0.019418330 0.040497602           0.02898837         9.173648
Sepal.Width  0.008553449 0.001962036 0.006951771           0.00575489         2.472105
Petal.Length 0.313303381 0.291818815 0.280981959           0.29216790        41.284869
Petal.Width  0.349686983 0.318527008 0.270975757           0.31054451        46.323415
> importance(fit)
               setosa versicolor virginica MeanDecreaseAccuracy MeanDecreaseGini
Sepal.Length 1.277324   1.632586  1.758101            1.2233029         9.173648
Sepal.Width  1.007943   0.252736  1.014141            0.6293145         2.472105
Petal.Length 3.685513   4.434083  4.133621            2.5139980        41.284869
Petal.Width  3.896375   4.421567  4.385642            2.5371353        46.323415
> 

I get different MeanDecreaseAccuracy values but have the same order for the importance Variables (for fit$importance as well as for importance(fit)):

  1. Petal.Width

  2. Petal.Length

  3. Sepal.Length

  4. Sepal.Width

But in other datasets I sometimes get different orders. Can someone explain what is happening here? Is this possibly a bug?


Edit (in response to Martin O'Leary)

Okay Thanks! I noticed something else.

Taking a Look at the rfcv() function i noticed the line:

impvar <- (1:p)[order(all.rf$importance[, 1], decreasing = TRUE)]

with this line we choose the first column of all.rf$importance which gives us the order of the class-specific (for the first factor) measures computed as mean descrease in accuracy only. This has not always the same order as the mean descrease in accuracy over all classes (MeanDecreaseAccuracy). Wouldn't it be better choosing either the MeanDecreaseAccuracy or MeanDecreaseGini column, or better using the importance()-function for the scaled values? So we would have a sequentially reduced number of predictors ranked by variable importance (over all classes) and not only ranked by variable importance for the first class.

$\endgroup$
12
$\begingroup$

No, this isn't a bug. The values given in fit$importance are unscaled, while the values given by importance(fit) are expressed in terms of standard deviations (as given by fit$importanceSD). This is usually a more meaningful measure. If you want the "raw" values, you can use importance(fit, scale=FALSE).

In general, it's a very bad idea to rely on the internal details of a fit object, when there's an extractor function provided. There are no guarantees as to the contents of fit$importance - they could change drastically from version to version without notice. You should always use the extractor function when it's provided.


Edit: Yes, that line in rfcv() does look like a bug, or at least unintended behaviour. It's actually quite a good example of why you shouldn't rely on the contents of things like fit$importance. If the fit is for a regression forest, the first column of fit$importance is %IncMSE, equivalent to importance(fit, type=1). However, this doesn't hold in the classification case, where you have extra columns for each factor level.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.