I have looked around and have seen GINI being used mostly in the context of binary classifiers. Does GINI make sense only for binary classifiers? Can we extend the definition to multiclass classifiers?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
The Gini impurity can definitely be used to quantify variance in a multi-class setting, not only in the binary case. Gini impurity is defined as
$$ G(p) = \sum_{i=1}^{J}{p_i} \sum_{k \neq i}^{J}{p_k} = 1-\sum_{i=1}^{J}{(p_i)^{2}} $$
for the scenario with $J$ classes, each having probability $p_i...p_J$, where $|J|$ can be $>2$.
More information can also be found here: https://en.wikipedia.org/wiki/Decision_tree_learning#Gini_impurity
Possibly you had a difficult time finding help due to the ambiguity with the "Gini Coefficient" used in economics (https://en.wikipedia.org/wiki/Gini_coefficient).
-
$\begingroup$ I just came across this answer and had to ask - must J be required to be greater than 2? Why will this not work if J = 2? $\endgroup$– TommyixiJan 18, 2020 at 21:55
-
3$\begingroup$ Hi @Tommyixi thanks for the question. The OP asked for Gini in a multiclass setting. Hence, the answer states that |J| can be > 2. Of course, |J| can also be 2. Hence, |J| >= 2 in the general case. Hope this makes sense. $\endgroup$– SimonMay 7, 2020 at 20:01