In the GBM package one is supposed to be able to provide interaction.depth>2, which means higher-order interactions between features. However, the resulting trees (as seen by pretty.gbm.tree) never show such interactions (and indeed - each row corresponds to just a single feature).

I'm not even sure anymore that the package actually supports depth>2....

Does anybody have any idea?


GBM considers the tree itself to be the instrument of realising interactions, thus interaction.depth equal $d$ means GBM will be building trees no deeper than $d$ rather than tress with splits considering $d$ attributes.

So, it works as expected.

  • $\begingroup$ Thanks a lot for your help, I've been googling for hours, and reading extensively 'elements of statistical learning', to no avail. Thanks a lot.. One more thing while we're at it, if you don't mind - are the trees weighted? or do I just average them out to get my prediction? $\endgroup$
    – zorbar
    Dec 21 '12 at 15:02
  • $\begingroup$ This is boosting, so their predictions are added as they are. They are only modified in case of shrinkage, but then all weights are the same so this is rather a normalisation. $\endgroup$
    – user88
    Dec 21 '12 at 15:37
  • $\begingroup$ @mbq in AdaBoost, the final prediction is given by weighted majority vote, i.e., the most accurate trees are given more weight. In Gradient Boosting, the votes of each tree are multiplied by the learning rate, as you said. But are they also weighted based on tree accuracy like in AdaBoost? $\endgroup$
    – Antoine
    Apr 26 '15 at 19:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.