When a metric is computed for a trained model with
nfolds greater than 1 in h2o, what is the value when `xval=TRUE' ?
Here is my read and please correct me if I'm off. If
nfolds is 3, it is doing 3-fold cross validation to select the best model. So, the
mse(train=T) gives the MSE of the best fold of the three models evaluated on the training set. Likewise,
valid=TRUE gives the error on the validation set by the same model.
But what is the MSE when xval is selected?
According to the
h2o.mse help page,
xval is the cross-validation error. So I expect it is the average of the 3 cross-validation runs.
However, when i look at the actual values,
xval is always lower than
valid, which contradicted my interpretation above. The best model must have a lower MSE than the average.