How can one see in a Boosted trees classification model, which variables interact with each other and how much? I would like to make use o this in R gbm package if possible
From this tutorial. See section 8 in particular.
Look up ?gbm.interactions. First construct your model, named angaus.tc5.lr005 in the tutorial.
angaus.tc5.lr005 <- gbm.step(data=Anguilla_train, gbm.x = 3:13, gbm.y = 2,
+ family = "bernoulli", tree.complexity = 5,
+ learning.rate = 0.005, bag.fraction = 0.5)
Then you will calculate the interactions in the model:
find.int <- gbm.interactions(angaus.tc5.lr005)
After this, you can access multiple attributes of your interaction, including the strength (
$interaction) and the rank (
Have a look through the tutorial, I think it will answer most of your questions. I did not write it.
Edit: This may be out of date as of at least May 2018, please see the comments below.
Additionally, you might also look at ?interact.gbm from the gbm package which implements Friedman's (2005) approach for detecting interactions.
J.H. Friedman and B.E. Popescu (2005). “Predictive Learning via Rule Ensembles.” Section 8.1