How can one see in a Boosted trees classification model, which variables interact with each other and how much? I would like to make use o this in R gbm package if possible


From this tutorial. See section 8 in particular.

Look up ?gbm.interactions. First construct your model, named angaus.tc5.lr005 in the tutorial.

angaus.tc5.lr005 <- gbm.step(data=Anguilla_train, gbm.x = 3:13, gbm.y = 2, + family = "bernoulli", tree.complexity = 5, + learning.rate = 0.005, bag.fraction = 0.5)

Then you will calculate the interactions in the model:

find.int <- gbm.interactions(angaus.tc5.lr005)

After this, you can access multiple attributes of your interaction, including the strength ($interaction) and the rank ($rank.list).

Have a look through the tutorial, I think it will answer most of your questions. I did not write it.

Edit: This may be out of date as of at least May 2018, please see the comments below.

  • $\begingroup$ any idea if does scikit-learn does this? $\endgroup$ – rraadd88 May 9 '17 at 7:16
  • $\begingroup$ @rraadd88 Sorry, I've never used it. You might want to ask a question here or ask the mailing list. $\endgroup$ – Chris C May 9 '17 at 14:42
  • $\begingroup$ gbm.interactions() from dismo does not appear to be compatible with gbm models created with the gbm package. See answer which suggests interact.gbm() from gbm. interact.gbm() only allows for the comparison of two variables at a time, however. $\endgroup$ – user29609 May 9 '18 at 23:47
  • $\begingroup$ @user29609 Thanks for the correction, I remember testing my answer to make sure it at least ran when I posted, perhaps the packages have gone out of sync? A quick look finds ?interact.gbm in the latest version of the gbm documentation which presumably will still be correct but I can't vouch for it. I'll put a note in my answer referring to your comment. $\endgroup$ – Chris C May 10 '18 at 8:07

Additionally, you might also look at ?interact.gbm from the gbm package which implements Friedman's (2005) approach for detecting interactions.

J.H. Friedman and B.E. Popescu (2005). “Predictive Learning via Rule Ensembles.” Section 8.1


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.