The caret package can help you optimize the parameter choice for your problem. The caretTrain vignette shows how to tune the gbm parameters using 10-fold repeated cross-validation - other optimization approaches are available it can all run in parallel using the foreach package. Use vignette("caretTrain", package="caret")
to read the document.
The package supports tuning shrinkage
, n.trees
, and interaction.depth
parameters for the gbm model, though you can add your own.
For heuristics, this is my initial approach:
shrinkage
: As small as you have time for (the gbm manual has more on this, but in general you can nver go wrong with a smaller value). Your data set is small so I'd probably start with 1e-3
n.trees
: I usually grow an initial model adding more and more trees until gbm.perf
says I have enough (actually, typically to 1.2 times that value) and then use that as a guide for further analysis.
interaction.depth
: you already have an idea about this. Try smaller values as well. Maximum value is floor(sqrt(NCOL(data)).
n.minobsinnode
: I find it really important to tune this variable. You don't want it so small that the algorithm finds too many spurious features.