I am trying to build a boosting model using the package gbm in R. I have the following code:

gb = gbm(aaa_target ~ .,

and when I have trained the model, I can get a summary like this:


The issue I am having, is that only a single variable (out of around 30) is selected and is given 100% predictive power. I know for a fact that many of the variables carry information (although the selected one is the most significant one), and using the randomForest package gives me a model which assigns significance to many of the variables.

Does anybody have a clue to why this might be the case?


Because the overworked maintainers of the gbm package have not had time to implement random feature sampling at each split calculation yet. I submitted a bad patch that did this as a proof of concept, but:

  • My C++ skills are non existent
  • I provided no documentation
  • I didn't integrate with the formula interface wrapper

So I feel no ill will for not picking up the patch. I haven't maintained the fork either so I'm sure it wouldn't integrate with the current gbm. You can see where I left off here: https://code.google.com/r/sheaparkes-mtry-additions/source/browse

If you really need the feature sampling functionality, it's available in Python's scikit.learn package implementation of gbm.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.