0
$\begingroup$ 
GBMmodel = gbm(mydataset~x1+x2+x3+x4+x5,
data=mydataset,distribution="gaussian",n.trees=1500,shrinkage=0.005,interaction.depth=3, bag.fraction=0.75,train.fraction=0.75,n.minobsinnode=5,cv.folds=3,keep.data=TRUE,verbose=TRUE)

Predicts = predict.gbm(...)

Then, we can obtain the "Predicts". I make R2 between mydataset and Predicts. I see, in this case, interaction.depth=3, R2 is about 0.7; if we set interaction.depth=5, R2 is about 0.8. So, how to specify interaction depth? It seems that interaction.depth is more, the fitted result is better. Why? And which interaction depth should be specified in GBM? choose 10? 20? ...

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ Are you evaluating the R2 on the training sample or on an test set ? $\endgroup$
    – RUser4512
    Feb 16, 2016 at 10:32
  • 1
    $\begingroup$ If you are evaluating performance on the training set, a larger interaction depth is always going to give a better $R^2$. Do you know about out of sample testing, and the dangers of in sample model evaluation? $\endgroup$
    – Matthew Drury
    Feb 17, 2016 at 2:47
  • $\begingroup$ Use caret R package optimization, this wrapper optimizes tuning parameters of many classifiers: stats.stackexchange.com/a/287093/3041 $\endgroup$
    – 42n4
    Jun 24, 2017 at 9:17

2 Answers 2

Reset to default
0
$\begingroup$

This is where the practitioner is important. You could make the problem better or worse with depth. There is a factorial explosion to be had, which is a bad thing. There is better feature accounting, and potential for higher fidelity, which would be good.

You can grid-search for parameters if you have a well set up measure of goodness.
You can also bring in domain knowledge, and it can have very good performance.

Make sure to consider the business goal, good math, and resource constraints that you have.

Improve this answer
$\endgroup$
0
$\begingroup$

The typical approach is to set hyperparameters such as the maximum tree depth etc. so as to optimize a suitable metric (depends on your context) on an appropriate validation set (or set of appropriate cross-validation splits) (i.e. split so as to match - as closely as possible - the prediction situation you want to use the model in). However, note that to some degree you can trade-off different things vs. each other (e.g. to some degree - once you have enough depth to represent relevant interactions - deeper trees can be traded off vs. more trees).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.