0
$\begingroup$

I've read many related articles and posts. The more I read, the more I got confused about 'Gini index' and 'Gini Impurity'. I understood the concept but it seems to me that these things are used differently by different people. ISLR book* (page 326) defines Gini Index as $\sum p_i(1– p_i)$ or $1 - \sum p_i^2$.

enter image description here

However, this (and many other articles) [the Same question has been asked in comments too by Shanu_not answered though] compute Gini by $ p^2+q^2$ formula for Binary classifier.

So, their Gini Impurity [ 1 $-$ Gini Index] is exactly the same as the Gini Index computed as per ISLR book.

Please let me know what am I missing. I realize that reading concepts after a long break is painful.

*Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani. (2013). An introduction to statistical learning : with applications in R. New York :Springer,

$\endgroup$
  • 1
    $\begingroup$ Apart from looking at the formulas, the words purity and impurity are indicative (so long as they are used carefully). $\sum p_i^2$ is maximal (purity is highest) when there is just one category present and so the sum is the sum of $1^2$ and any number of $0^2$ and so just $1$. $1 - \sum p_i^2$ is minimal in the same case (impurity is lowest). $\endgroup$ – Nick Cox Jun 5 '19 at 6:09
  • 1
    $\begingroup$ $p^2 + q^2$ is clearly not a general recipe, but applies only when there are two categories in play. $\endgroup$ – Nick Cox Jun 5 '19 at 6:12
  • $\begingroup$ The Gini index formula is the $G$ you defined above. That $p^2 + q^2$ computes somehow purity, it is specific to two classes, and the $1$ from $G$ got removed because it is constant when you compare two nodes in a decision tree. Usually splitting criteria in decision trees use impurity measures: eg Gini index or entropy. An example here: stats.stackexchange.com/questions/44382/… $\endgroup$ – Simone Jun 5 '19 at 6:29
  • $\begingroup$ ISLR won't be recognisable by all readers, so please give references in good academic style (authors, date, book title, publishers, place). $\endgroup$ – Nick Cox Jun 5 '19 at 7:44
  • $\begingroup$ Thanks Nick, done that. $\endgroup$ – Dr Nisha Arora Jun 5 '19 at 16:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.