My model is working ok (the AUC is 0.7) but the importances from a randomForest
run for my binary classification problem differ depending on how I retrieve them. Is this normal? They seem to be scaled up when I call importance(rf)
(by order 105). More importantly, the first three columns don't match at all, even discounting the magnitude! I'm unsure how to interpret this at the moment having read ?importance
and ?randomForest
.
Here's my randomForest call:
rf <- randomForest(trnX, as.factor(trnY), ntree=1000, importance=TRUE)
and here's a portion of the outputs:
rf$importance
:
0 1 MeanDecreaseAccuracy MeanDecreaseGini
var1 -2.308793e-05 -6.124117e-05 -3.642557e-05 30.47050
var2 6.169346e-04 -3.947637e-04 2.563570e-04 48.04932
var3 3.621287e-03 1.834355e-03 2.981152e-03 68.28302
var4 3.655234e-03 1.981978e-03 3.057267e-03 79.12254
var5 5.350649e-03 1.041555e-03 3.812376e-03 84.88832
var6 3.366199e-03 1.707144e-03 2.773894e-03 78.02293
...
importance(rf)
:
0 1 MeanDecreaseAccuracy MeanDecreaseGini
var1 -0.4739717 -0.8497967 -0.8952541 30.47050
var2 7.9028043 -3.6680758 4.1625329 48.04932
var3 17.1279919 5.9733456 20.8354079 68.28302
var4 17.3895593 7.2370419 22.4948931 79.12254
var5 20.7470555 3.2702612 24.3069376 84.88832
var6 16.9071520 6.2532409 21.1648686 78.02293
...
My data has 8000 data points and 60 variables. There is definitely correlation between variables and they are of different types: some are binary flags, other are integers, others continuous. Some variables have missing values (which I'm fixing with na.roughfix()
).
Additionally, a run from gbm:
gbmBernModel <- gbm(formula = var1 ~ ., distribution="bernoulli",
data = trn, verbose="CV", n.trees=10000, interaction.depth=2)
produced relative importances (viewed by summary(gbmBernModel)
) that made logical sense to me.
Any pointers would be greatly appreciated. Thank you.