How many models does gbm fit?

Question

In gbm funtion, if we set train.fraction=0.5, cv.fold=5. Does gbm fit twice for trained sample and CV sample,respectively? 1) 50% of sample is used as trained data to fit the 1st model. 2) 80% of sample is used as CV sample to fit the second model

If that is the case, why gbm.perf only reports the same error curve for both the cases? The only thing differs is just the best iteration number....

If not the case, how does gbm work behind the scene?

Thank you very much!

Answer 1

In this situation you will fit:

• Five full gradient boosted models, each on $\frac{4}{5}$ of the available training data. Each tree trained within these five boosters has access to all and only $\frac{4}{5}$ of the training data, and it's the same $\frac{4}{5}$ for each tree.
• Within each of these five boosted models, each tree will be trained on a different $\frac{1}{2}$ of the in-fold training data. This works out to $\frac{4}{5} \times \frac{1}{2} = \frac{2}{5}$ of the total available training data.

The subsampling only serves to diversify the trees fit in a single boosted model, it plays no role in evaluating the model. The curves pictured are the average out-of-fold error, with the average taken over the five full boosted models created during cross validation.