I have looked around and have seen GINI being used mostly in the context of binary classifiers. Does GINI make sense only for binary classifiers? Can we extend the definition to multiclass classifiers?


The Gini impurity can definitely be used to quantify variance in a multi-class setting, not only in the binary case. Gini impurity is defined as

$$ G(p) = \sum_{i=1}^{J}{p_i} \sum_{k \neq i}^{J}{p_k} = 1-\sum_{i=1}^{J}{(p_i)^{2}} $$

for the scenario with $J$ classes, each having probability $p_i...p_J$, where $|J|$ can be $>2$.

More information can also be found here: https://en.wikipedia.org/wiki/Decision_tree_learning#Gini_impurity

Possibly you had a difficult time finding help due to the ambiguity with the "Gini Coefficient" used in economics (https://en.wikipedia.org/wiki/Gini_coefficient).

  • $\begingroup$ I just came across this answer and had to ask - must J be required to be greater than 2? Why will this not work if J = 2? $\endgroup$ – Tommyixi Jan 18 '20 at 21:55
  • 1
    $\begingroup$ Hi @Tommyixi thanks for the question. The OP asked for Gini in a multiclass setting. Hence, the answer states that |J| can be > 2. Of course, |J| can also be 2. Hence, |J| >= 2 in the general case. Hope this makes sense. $\endgroup$ – Simon May 7 '20 at 20:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.