Why we are using squared probabilities instead of normal probabilities in Gini impurity . Probabilities will always be positive, so why to square those?

  • 3
    $\begingroup$ @Tim The Gini Impurity formula that you linked to can be rewritten as $1-\Sigma_i^C p(i)^2$, which does use them. $\endgroup$
    – dimitriy
    Aug 3, 2020 at 23:47
  • $\begingroup$ @Daya Do you want a math-y explanation? Do you seek intuition? Is there a particular area, like classification with trees, where you are using GI that would make a good example? $\endgroup$
    – dimitriy
    Aug 4, 2020 at 0:06
  • $\begingroup$ @DimitriyV.Masterov , Yes i want to know math explanation for that equation , it would be great helpul if you can explain or provide some link where i can get my answet $\endgroup$
    – Daya
    Aug 4, 2020 at 1:47
  • $\begingroup$ I think Tim's link has good intuition. There are some older, mathier questions that you can discover by searching this site, such as this one. $\endgroup$
    – dimitriy
    Aug 4, 2020 at 3:20
  • $\begingroup$ stats.stackexchange.com/questions/473702/… $\endgroup$ Aug 4, 2020 at 22:06

1 Answer 1


The above answers are all excellent. When I had the same question, I managed to get an intuition of this by simply doing the following

temp= []
for j in range(0, 10):
    i = j / 10.0
    num1 = i * i
    num2 = (1-i) * (1-i)
    temp.append(num1 + num2)

temp = [1.0, 0.8200000000000001, 0.6800000000000002, 0.58, 0.52, 0.5, 0.52, 0.58, 0.6800000000000002, 0.8200000000000001] 

As you can see, the sum of squares minimizes when at least one of the probabilities goes towards extreme values (0 and 1 being extremes). In Gini impurity, that is what we want - we want to split the node which results in the probabilities of 2 classes being extreme. i.e. one split should have only members of class A and another split members of class B (if this was a 2-class problem).
As you can see form the above, that is achieved when you maximize the sum of squares of probabilities.


Not the answer you're looking for? Browse other questions tagged or ask your own question.