I am working with the GBM algorithm in H2O in R. I am using 100% of the data as the training data, and then using 5-fold cross-validation to train and validate my model using 100% of the data.
My question, then, is how are the training metrics and the cross-validation metrics calculated? I'm fairly new to all this, so my understanding may be limited and I'm trying to wrap my head around these things.
I took a look at the description here: http://docs.h2o.ai/h2o/latest-stable/h2o-docs/cross-validation.html, and it seems to describe how the cross-validation metrics are calculated but not the training metrics:
"For the main model, this is how the cross-validation metrics are computed: The 5 holdout predictions are combined into one prediction for the full training dataset (i.e., predictions for every row of the training data, but the model making the prediction for a particular row has not seen that row during training). This “holdout prediction” is then scored against the true labels, and the overall cross-validation metrics are computed."
So is it the case that, to calculate the cross-validation metrics, the model will combine the predictions on each (i.e., n=5) held-out group of observations, and then compare those to the actual observed values? In that case, how are the training metrics computed? Any explanation would be helpful!