# logloss vs gini/auc

I've trained two models (binary classifiers using h2o AutoML) and I want to select one to use. I have the following results:

 model_id        auc     logloss    logloss_train   logloss_valid   gini_train  gini_valid

DL_grid_1   0.542694    0.287469         0.092717        0.211956     0.872932    0.312975
DL_grid_2   0.543685    0.251431         0.082616        0.186196     0.900955    0.312662


the auc and logloss columns are the cross-validation metrics (the cross validation only uses the training data). the ..._train and ..._valid metrics are found by running the training and validation metrics through the models respectively. I want to either use the logloss_valid or the gini_valid to choose a the best model.

Model 1 has a better gini (i.e. better AUC) but model two has a better logloss. My question is which one to choose which I think begs the question, what are the advantages/disadvantages to using either gini (AUC) or logloss as a decision metric.

• This video has a nice explanation of why logloss is preferred if you're interested in the probabilities and not just the classification. Note that for binary classification, logloss is equal to the brier score. – Dan Jul 23 at 10:20

Whereas the AUC is computed with regards to binary classification with a varying decision threshold, logloss actually takes "certainty" of classification into account.

Therefore to my understanding, logloss conceptually goes beyond AUC and is especially relevant in cases with imbalanced data or in case of unequally distributed error cost (for example detection of a deadly disease).

In addition to this very basic answer, you might want to have a look at optimizing auc vs logloss in binary classification problems

A simple example of logloss computation and the underlying concept is discussed in this recent question Log Loss function in scikit-learn returns different values