Gini index/criterion/impurity/coefficient according to Breiman, Sen, Bishop, and Duda

Question

once again a student (me) is lost in the sea of Gini... I am currently trying to figure out, where the Gini based formula for feature selection proposed by Cehovin and Bosnic [1] comes from:

$Gini(A)=\sum_{j}p(j)\sum_{k}p(k|j)^2 - \sum_{k}p(k)^2 $,

with $p(j)$ being the probability that the feature $A$ takes value $j$, $p(k|j)$ being the probability that the sample is of class $k$ given feature $A$ has value $j$, and $p(k)$ being the probability that a random sample belongs to class $k$.

Is it right, that ...

• The formula uses the Gini "criterion" according to Breiman et al.[2] ($1-\sum_k p_{ik}^2$)?
• The right side (right of the $-$) is the Gini "criterion" [2] of the original dataset.
• The left side (left side of $-$) refers the weighted average of the Gini "criterions" (averaged by the probability of feature $A$ taking the respective value $j$, see $\sum_{j}p(j))$. The average is calculated across all "sub" data sets, which each sub data set holding only samples of one distinct value $j$ of feature $A$. Thus, $\sum_{k}p(k|j)^2$ essentially refers the Gini "criterion" for the sub data set.
• What is the meaning of the $\sum_{j \neq k} $ formulation e.g. Breiman et al. 2 are using?
• All of this (Gini "criterion" Breiman [2], Gini "index" Bishop [3], Gini "impurity" Duda [4]) has (mathematically) absolutely nothing to do with the Gini "coefficient" proposed by Sen [5].

Do I understand the meaning of this formula correctly? Can somebody shed some light on this confusion topic?

Many thanks

[1] Cehovin and Bosnic, Empirical evaluation of feature selection methods in classification

[2] Breiman, Classification and Regression Trees (CART)

[3] Bishop, Pattern Recognition and Machine Learning

[4] Duda, Pattern Recognition

[5] Sen, On Economic Inequqlity

Answer 1

I am the first author of [1], the work is almost ten years old now (done during my master's degree for one of the courses) so I do not remember everything, but what I can say is that we did not "propose" the Gini criterion for feature selection, at that time we have used the definition as described in "Machine Learning and Data Mining" by Kononenko and Kukar. For what I know they did not propose it either, but simply mention it (the book does not make any clear reference who proposed it). In the book the definition is indeed linked to Gini "criterion" [2]. I cannot reliably comment on the rest of the questions as my work is nowadays no longer pure machine learning.