Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
What are the pros and cons of using Deviance as opposed to Gini coefficient when measuring the quality of regression / classification models?
From experience, I see that people like Gini more than Deviance. I don't know the reason, but perhaps the Deviance is too sensible to the outlier, and the Gini is not. It can be inconvenient our convenient at the same time. For me, both measures should be considered at the same time.
As also mentioned in the link Scortchi supplies the Gini coefficient (or the proportional c-statistic or AUC) only contains information how well the model ranks the outcomes and no information about the calibration.
The deviance in a binary glm model is going twice the negative value of logarithmic scoring rule as shown here
> model <- glm(formula= vs ~ wt + disp, data=mtcars, family=binomial)
>
> # the negative deviance
> -model$deviance
[1] -21.40039
>
> # the logarithmic scoring rule
> ps <- predict(model, type = "response")
> with(mtcars, sum(vs * log(ps) + (1 - vs) * log(1 - ps)))
[1] -10.70019
The logarithmic scoring rule does provide information about the calibration. You may also want to see this post.
