I've trained two models (binary classifiers using h2o AutoML) and I want to select one to use. I have the following results:
model_id auc logloss logloss_train logloss_valid gini_train gini_valid DL_grid_1 0.542694 0.287469 0.092717 0.211956 0.872932 0.312975 DL_grid_2 0.543685 0.251431 0.082616 0.186196 0.900955 0.312662
logloss columns are the cross-validation metrics (the cross validation only uses the training data). the
..._valid metrics are found by running the training and validation metrics through the models respectively. I want to either use the
logloss_valid or the
gini_valid to choose a the best model.
Model 1 has a better gini (i.e. better AUC) but model two has a better logloss. My question is which one to choose which I think begs the question, what are the advantages/disadvantages to using either gini (AUC) or logloss as a decision metric.