In the decision tree based classification technique. What is the difference among the different approaches like entropy, gini index? When to use entropy and when to use gini index?.

  • 1
    $\begingroup$ This question appears to be off-topic because it is about math. $\endgroup$ – Sean Owen Oct 2 '13 at 19:07
  • 1
    $\begingroup$ I would use entropy if you're not sure. One advantage of entropy is that it is also defined for continuous variables, but not so for Gini impurity. $\endgroup$ – Sean Owen Oct 2 '13 at 19:08
  • 2
    $\begingroup$ @SeanOwen, then again, everything is about math. $\endgroup$ – Don Reba Oct 2 '13 at 19:40
  • $\begingroup$ Sure, at some level, but at the level that StackExchange divides up, questions here aren't generally about math. math.stackexchange.com will get much better answers, although for some reason there's no option to close and move it there. $\endgroup$ – Sean Owen Oct 2 '13 at 20:30

Here is a quote from a paper on the subject:

Different split criteria were proposed in the literature (Information Gain, Gini Index, etc.). It is not obvious which of them will produce the best decision tree for a given data set. A large amount of empirical tests were conducted in order to answer this question. No conclusive results were found.

Raileanu, L. E., & Stoffel, K. (2004). Theoretical Comparison between the Gini Index and Information Gain Criteria. Annals of Mathematics and Artificial Intelligence, 41(1), 77-93. Kluwer Academic Publishers.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy