How can one see in a Boosted trees classification model, which variables interact with each other and how much? I would like to make use o this in R gbm package if possible
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
4
From this tutorial. See section 8 in particular.
Look up ?gbm.interactions. First construct your model, named angaus.tc5.lr005 in the tutorial.
angaus.tc5.lr005 <- gbm.step(data=Anguilla_train, gbm.x = 3:13, gbm.y = 2,
+ family = "bernoulli", tree.complexity = 5,
+ learning.rate = 0.005, bag.fraction = 0.5)
Then you will calculate the interactions in the model:
find.int <- gbm.interactions(angaus.tc5.lr005)
After this, you can access multiple attributes of your interaction, including the strength ($interaction) and the rank ($rank.list).
Have a look through the tutorial, I think it will answer most of your questions. I did not write it.
Edit: This may be out of date as of at least May 2018, please see the comments below.
-
$\begingroup$ any idea if does scikit-learn does this? $\endgroup$– user156469Commented May 9, 2017 at 7:16
-
$\begingroup$ @rraadd88 Sorry, I've never used it. You might want to ask a question here or ask the mailing list. $\endgroup$– Chris CCommented May 9, 2017 at 14:42
-
$\begingroup$
gbm.interactions()fromdismodoes not appear to be compatible with gbm models created with thegbmpackage. See answer which suggestsinteract.gbm()fromgbm.interact.gbm()only allows for the comparison of two variables at a time, however. $\endgroup$ Commented May 9, 2018 at 23:47 -
$\begingroup$ @user29609 Thanks for the correction, I remember testing my answer to make sure it at least ran when I posted, perhaps the packages have gone out of sync? A quick look finds
?interact.gbmin the latest version of thegbmdocumentation which presumably will still be correct but I can't vouch for it. I'll put a note in my answer referring to your comment. $\endgroup$– Chris CCommented May 10, 2018 at 8:07
$\begingroup$
$\endgroup$
Additionally, you might also look at ?interact.gbm from the gbm package which implements Friedman's (2005) approach for detecting interactions.
J.H. Friedman and B.E. Popescu (2005). “Predictive Learning via Rule Ensembles.” Section 8.1
