4
$\begingroup$

In the GBM package one is supposed to be able to provide interaction.depth>2, which means higher-order interactions between features. However, the resulting trees (as seen by pretty.gbm.tree) never show such interactions (and indeed - each row corresponds to just a single feature).

I'm not even sure anymore that the package actually supports depth>2....

Does anybody have any idea?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

GBM considers the tree itself to be the instrument of realising interactions, thus interaction.depth equal $d$ means GBM will be building trees no deeper than $d$ rather than tress with splits considering $d$ attributes.

So, it works as expected.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks a lot for your help, I've been googling for hours, and reading extensively 'elements of statistical learning', to no avail. Thanks a lot.. One more thing while we're at it, if you don't mind - are the trees weighted? or do I just average them out to get my prediction? $\endgroup$
    – zorbar
    Dec 21, 2012 at 15:02
  • $\begingroup$ This is boosting, so their predictions are added as they are. They are only modified in case of shrinkage, but then all weights are the same so this is rather a normalisation. $\endgroup$
    – user88
    Dec 21, 2012 at 15:37
  • $\begingroup$ @mbq in AdaBoost, the final prediction is given by weighted majority vote, i.e., the most accurate trees are given more weight. In Gradient Boosting, the votes of each tree are multiplied by the learning rate, as you said. But are they also weighted based on tree accuracy like in AdaBoost? $\endgroup$
    – Antoine
    Apr 26, 2015 at 19:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.