As the title says, I'm getting some interesting results from gbm.perf. The first time I ran into trouble was after a run where n.trees was set to 7,000. When gbm.perf also returned 7,000 I got suspicious. So I ran a test where n.trees was 7. gbm.perf's value: 7.
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
1
That's not all that unusual. What typically happens with boosting is that it can take many iterations before the cross-validated loss function bottoms out. Indeed, depending on the model that you are trying to fit, you may require thousands if not tens of thousands of trees before it hits that point. If you've got the computational resources, it would be well worth exploring the space in detail. gbm.perf will return whichever iteration stage it records as having the lowest loss function.
The gbm documentation has some very interesting examples relating to the relationship between shrinkage, loss function and the number of trees required to reach the minimum CV loss function. Check out page 8.
http://gradientboostedmodels.googlecode.com/git/gbm/inst/doc/gbm.pdf
-
$\begingroup$ The link is not valid. It can be accessed by typing utils::browseVignettes("gbm") (in R) $\endgroup$– UriNov 14 at 10:44
$\begingroup$
$\endgroup$
It depends on your shrinkage. A small learning-rate would take more iterations to achieve minimal loss.
