Gradient boosting in R uses only a single variable

Question

I am trying to build a boosting model using the package gbm in R. I have the following code:

gb = gbm(aaa_target ~ .,
     data=myDdata,
     n.trees=100,
     verbose=TRUE)

and when I have trained the model, I can get a summary like this:

summary(gb)

The issue I am having, is that only a single variable (out of around 30) is selected and is given 100% predictive power. I know for a fact that many of the variables carry information (although the selected one is the most significant one), and using the randomForest package gives me a model which assigns significance to many of the variables.

Does anybody have a clue to why this might be the case?

Answer 1

Because the overworked maintainers of the gbm package have not had time to implement random feature sampling at each split calculation yet. I submitted a bad patch that did this as a proof of concept, but:

• My C++ skills are non existent
• I provided no documentation
• I didn't integrate with the formula interface wrapper

So I feel no ill will for not picking up the patch. I haven't maintained the fork either so I'm sure it wouldn't integrate with the current gbm. You can see where I left off here: https://code.google.com/r/sheaparkes-mtry-additions/source/browse

If you really need the feature sampling functionality, it's available in Python's scikit.learn package implementation of gbm.