# Gini index/criterion/impurity/coefficient according to Breiman, Sen, Bishop, and Duda

once again a student (me) is lost in the sea of Gini... I am currently trying to figure out, where the Gini based formula for feature selection proposed by Cehovin and Bosnic [1] comes from:

$Gini(A)=\sum_{j}p(j)\sum_{k}p(k|j)^2 - \sum_{k}p(k)^2$,

with $p(j)$ being the probability that the feature $A$ takes value $j$, $p(k|j)$ being the probability that the sample is of class $k$ given feature $A$ has value $j$, and $p(k)$ being the probability that a random sample belongs to class $k$.

Is it right, that ...

• The formula uses the Gini "criterion" according to Breiman et al.[2] ($1-\sum_k p_{ik}^2$)?
• The right side (right of the $-$) is the Gini "criterion" [2] of the original dataset.
• The left side (left side of $-$) refers the weighted average of the Gini "criterions" (averaged by the probability of feature $A$ taking the respective value $j$, see $\sum_{j}p(j))$. The average is calculated across all "sub" data sets, which each sub data set holding only samples of one distinct value $j$ of feature $A$. Thus, $\sum_{k}p(k|j)^2$ essentially refers the Gini "criterion" for the sub data set.
• What is the meaning of the $\sum_{j \neq k}$ formulation e.g. Breiman et al. 2 are using?
• All of this (Gini "criterion" Breiman [2], Gini "index" Bishop [3], Gini "impurity" Duda [4]) has (mathematically) absolutely nothing to do with the Gini "coefficient" proposed by Sen [5].

Do I understand the meaning of this formula correctly? Can somebody shed some light on this confusion topic?

Many thanks

[1] Cehovin and Bosnic, Empirical evaluation of feature selection methods in classification

[2] Breiman, Classification and Regression Trees (CART)

[3] Bishop, Pattern Recognition and Machine Learning

[4] Duda, Pattern Recognition

[5] Sen, On Economic Inequqlity