# How the Internal H2O auc measures are calculated? Why they are so close to 1 or 1?

I am randomly holding out 10% of data out of the whole dataset as test.data and train the GBM model on a remaining 90% of rows train.data (with x and y provided, no nfolds or validation data set provided )...

Once I print it, the trained model - it shows auc measure of 1 or 0.999.

But when I actually validate predictions against the initially held out 10% of data the auc comes back to around 0.91

I think I ruled out data leak, reviewed the code and looked at the variables importance with legit variables being appropriately important.

What is going on?

If you only provide H2O's GBM with a training_frame, then the only metrics it can produce is training metrics. That's why your AUC is so high -- it's training AUC. You should not assume that you training metrics will be similar to your test set metrics. Use h2o.performance(model, test.data) to generate test set metrics. If you want AUC specifically, then use h2o.auc(h2o.performance(model, test.data)).