2
$\begingroup$

In the building of a decision tree, when it's better to prefer the information gain criterion to the gain ratio criterion ? And why ?

$\endgroup$
3
  • $\begingroup$ The Wikipedia article answers your question. $\endgroup$ – deemel May 16 '18 at 8:57
  • $\begingroup$ It's explains the limits of the information gain, i wanna know instead when i should use it $\endgroup$ – Qwerto May 16 '18 at 9:13
  • $\begingroup$ Pardon my hasty comment, you're right. I'll try to provide an answer to it instead $\endgroup$ – deemel May 16 '18 at 13:47
3
$\begingroup$

If two attributes with different number of possible values (categories), have the same Enthropy, Info Gain cannot differentiate them (Decision tree algorithm will select one of them randomly). In the same situation Gain Ratio, will favor attribute with less categories.

Gain ratio strategy, leads to better generalization (less overfitting) of DT models and it is better to use Gain ration in general.

Even if one would like to favor attributes with more categories, Info Gain wouldn't be a good choice since it does not differentiate between attributes with different numbers of categories.

Hope this helps!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.