11
$\begingroup$

Kaggle's competition Porto Seguro's Safe Driver Prediction uses Normalized Gini Score as evaluation metric and this got me curious about the reasons for this choice. What are the advantages of using normalized gini score instead of the most usual metrics, like AUC, for evaluation?

$\endgroup$
  • 1
    $\begingroup$ The Kaggle website used to have this answer: "There is a maximum achievable area for a "perfect" model since not all of the positive examples occur immediately. We use the normalized Gini coefficient by dividing the Gini coefficient of your model by the Gini coefficient of the perfect model." but it is not available anymore. webcache.googleusercontent.com/… $\endgroup$ – Martijn Weterings Oct 10 '17 at 1:01
  • 1
    $\begingroup$ So, gini is just auc on a different scale. Or are auc and gini applied to different curves? That is not clear to me as non-expert in machine-learning. The question is not very clear about this. $\endgroup$ – Martijn Weterings Oct 10 '17 at 1:10
1
$\begingroup$

I believe that the Gini score is merely a reformulation of the AUC: $$gini = 2 \times AUC -1$$ As for why use this instead of the commonly used AUC, the only reason I can think of is that a random prediction will yield a Gini score of 0 as opposed to the AUC which will be 0.5.

$\endgroup$
  • 5
    $\begingroup$ Besides that using the gini coefficient sets the performance of a random classifier to a score of 0... the normalization "improves" the other end of the scale and makes that the score of a perfect classifier is equal to 1 rather than a maximum achievable AUC<1. The improvement being only relative depending on whether you think a more intuitive scale is good or not. Although beyond this easier interpretation you might argue that it (the normalization) also improves generalization and comparison of different data-sets. $\endgroup$ – Martijn Weterings Oct 10 '17 at 21:16
  • $\begingroup$ Why should macimum achievable AUC be less then 1, also I don't see how is gini setting it to 1? $\endgroup$ – rep_ho Oct 11 '17 at 14:21
  • $\begingroup$ It depends on what kind of curve they calculate the gini coefficient. Maybe they use something different than a ROC curve (whose max AUC would indeed be 1). Given the words on the kaggle website it seems plausible that the max AUC is not 1: > " We then move from left to right, asking "In the leftmost x% of the data, how much of the cumulative response have you accumulated?" $\endgroup$ – Martijn Weterings Oct 12 '17 at 15:44
  • $\begingroup$ example: bayesserver.com/docs/charts/lift-chart $\endgroup$ – Martijn Weterings Oct 12 '17 at 15:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.