# Interpreting GBM interact.gbm

I am learning GBM with a focus on the interactions side of things I am aware of the H statistic which ranges from 0-1 where large values indicate strong effects. I created a dummy experiment below using R. I predict the species type from the attributes in the Iris dataset.

library(caret)
library(gbm)
data(iris)
set.seed(5555)

gbmGrid <-  expand.grid(interaction.depth = 3,
n.trees = 100,
shrinkage = 0.1,
n.minobsinnode = (3:5))

trctrl <- trainControl(method = "repeatedcv",
number = 10, repeats = 3)

# 3 rows to tune
nrow(gbmGrid)

gbmModel <- train(Species~.,  data = iris,  method = "gbm",
trControl=trctrl, tuneGrid = gbmGrid, verbose = FALSE)

# Find the best iteration
best.iter <- gbm.perf(gbmModel$finalModel, method="OOB") interact.gbm(gbmModel$finalModel, data = iris, n.trees = best.iter,
i.var = c('Petal.Length','Petal.Width','Sepal.Length'))
#      setosa   versicolor    virginica
#0.0002366084 0.0413793842 0.0050366585


I have two questions

• I would just like to confirm my understanding for the final output. Is this saying that for predicting the versicolor species there is a relatively strong interaction between the Petal Length, Petal Width and Sepal.Length attributes. Comparatively this is not such an important interaction when predicting the virginica and setosa classes.
• In my code above i have specified two items i believe there is an interaction term. Is there a way to systematically test for many interactions, say for example as an exploratory exercise (then using a test set to confirm any interactions found). I have seen the package dismo which has the function gbm.interactions but this seems to be specific to regression trees.

## migrated from stackoverflow.comMay 6 '18 at 17:42

This question came from our site for professional and enthusiast programmers.

• Can you please check that you get the same results as the one I added when I ran your code? If not can you please add the H-statistics you observe? – usεr11852 May 7 '18 at 11:13
• You should be aware that the gbm package has been orphaned. – aginensky May 7 '18 at 11:19
• Hi @usεr11852, I can confirm your results. Thank you very much for your quick response. The interactions are small but i think for illustration they are OK. – John Smith May 7 '18 at 12:13
• The standard replacement is xgboost, which is also supported by caret. There are two other newer packages by yandex and microsoft - if I memory serves me. They really are improvements in that they are significantly faster than gbm. – aginensky May 7 '18 at 13:28
• Hi @aginensky, i had a look but i don't seem to be able to find anything specific to interactions. Could you point me in the right direction for xgboost; be that blogs or research papers that explore feature interactions? – John Smith May 7 '18 at 14:34