5
$\begingroup$

What are the pros and cons of using Deviance as opposed to Gini coefficient when measuring the quality of regression / classification models?

From experience, I see that people like Gini more than Deviance. I don't know the reason, but perhaps the Deviance is too sensible to the outlier, and the Gini is not. It can be inconvenient our convenient at the same time. For me, both measures should be considered at the same time.

$\endgroup$
  • 2
    $\begingroup$ See ROC curve drawbacks: the Gini coefficient is a function of the area under the ROC curve, & the deviance gives a (logarithmic) proper scoring rule. $\endgroup$ – Scortchi Jun 21 '16 at 8:36
1
$\begingroup$

As also mentioned in the link @Scortchi supplies the Gini coefficient (or the proportional c-statistic or AUC) only contains information how well the model ranks the outcomes and no information about the calibration.

The deviance in a binary glm model is going twice the negative value of logarithmic scoring rule as shown here

> model <- glm(formula= vs ~ wt + disp, data=mtcars, family=binomial)
> 
> # the negative deviance
> -model$deviance
[1] -21.40039
> 
> # the logarithmic scoring rule
> ps <- predict(model, type = "response")
> with(mtcars, sum(vs * log(ps) + (1 - vs) * log(1 - ps)))
[1] -10.70019

The logarithmic scoring rule does provide information about the calibration. You may also want to see this post.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.