I realise that there are 3 tuning parameters in the boosting trees model, i.e.
- the number of trees (number of iterations)
- shrinkage parameter
- number of splits (size of each constituent trees)
My question is: for each of the tuning parameters, how should I find its optimal value ? And what method ?
Note that: the shrinkage parameter and the number of trees parameter operate together, i.e. a smaller value for shrinkage parameter leads to a higher value for the number of trees. And we need to take this into account too.
I am particularly interested in the method to find the optimal value for the number of splits. Should it be based on cross-validation or domain knowledge about the model behind?
And how are these things carried out in the
gbm package in R ?