1
$\begingroup$

I created an cartesian grid of GBMs using h2o package in R and saved cross-validation metrics for each model in a data frame. So, for each model, I stored the results given in model@model$cross_validation_metrics_summary.

What is the threshold used to calculate F1 and F2 scores, precision, recall and specificity in model@model$cross_validation_metrics_summary? Is there a default value?

$\endgroup$

1 Answer 1

1
$\begingroup$

It appears the threshold is chosen to maximize the F1-score.

Prediction Threshold For classification problems, when running h2o.predict() or .predict(), the prediction threshold is selected as follows:

  • If you train a model with only training data, the Max F1 threshold from the train data model metrics is used.
  • If you train a model with train and validation data, the Max F1 threshold from the validation data model metrics is used.
  • If you train a model with train data and set the nfold parameter, the Max F1 threshold from the training data model metrics is used.
  • If you train a model with the train data and validation data and also set the nfold parameter, the Max F1 threshold from the validation data model metrics is used.

from http://docs.h2o.ai/h2o/latest-stable/h2o-docs/performance-and-prediction.html#prediction-threshold

and

For classification problems, predicted probabilities and labels are compared against known results. (Note that for binary models, labels are based on the maximum F1 threshold from the model object.)

from http://docs.h2o.ai/h2o/latest-stable/h2o-docs/performance-and-prediction.html#prediction

$\endgroup$
1
  • $\begingroup$ Thanks! I searched for this in the documentation and didn't found it. $\endgroup$
    – user260147
    Sep 17, 2019 at 20:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.