I am learning
GBM with a focus on the interactions side of things
I am aware of the H statistic which ranges from 0-1 where large values indicate strong effects. I created a dummy experiment below using R.
I predict the species type from the attributes in the Iris dataset.
library(caret) library(gbm) data(iris) set.seed(5555) gbmGrid <- expand.grid(interaction.depth = 3, n.trees = 100, shrinkage = 0.1, n.minobsinnode = (3:5)) trctrl <- trainControl(method = "repeatedcv", number = 10, repeats = 3) # 3 rows to tune nrow(gbmGrid) gbmModel <- train(Species~., data = iris, method = "gbm", trControl=trctrl, tuneGrid = gbmGrid, verbose = FALSE) # Find the best iteration best.iter <- gbm.perf(gbmModel$finalModel, method="OOB") interact.gbm(gbmModel$finalModel, data = iris, n.trees = best.iter, i.var = c('Petal.Length','Petal.Width','Sepal.Length')) # setosa versicolor virginica #0.0002366084 0.0413793842 0.0050366585
I have two questions
- I would just like to confirm my understanding for the final output. Is this saying that for predicting the versicolor species there is a relatively strong interaction between the Petal Length, Petal Width and Sepal.Length attributes. Comparatively this is not such an important interaction when predicting the virginica and setosa classes.
- In my code above i have specified two items i believe there is an interaction term. Is there a way to systematically test for many interactions, say for example as an exploratory exercise (then using a test set to confirm any interactions found). I have seen the package dismo which has the function
gbm.interactionsbut this seems to be specific to regression trees.