# Why decision tree handle unbalanced data well?

one approach to deal with the unbalanced dataset is to choose the models that can hand this type of dataset well such as decision tree, but why decision tree can handle the unbalanced dataset well?

Decision trees do not always handle unbalanced data well.

If there is relatively obvious particular partition of our sample space that contains a high-proportion of minority class instances, decision trees can probably find it but that is far from a certainty. For example, if the minority class is strongly associated with multiple features interacting with each other, it is rather demanding for a tree to recognise the pattern; even if it does, it will probably be a rather deep and unstable tree that will be prone to over-fitting; pruning the tree will not immediately solve the problem because it will directly affect the ability of the tree to utilise those interactions.

Generalisations of decision tree algorithms as random forests and gradient boosting machines offer a much better alternative in terms of stability without sacrificing any performance. Similarly using a GAM with an interacting spline can also provide another potentially viable alternative.

• Absolutely agree that decision trees don't inherently solve the class imbalance problem. The optimizing function itself will have a large impact - if your decision tree is trained to optimize accuracy, for example, it will handle class imbalance much worse than a decision tree that optimizes the F1 score or AUC. Feb 25, 2020 at 21:47
• @NuclearWang: I agree (+1); I assume that the "standard decision tree" is the CART so we use Gini impurity by default. That said, even if we employ a "good measure" we might still over-fit and experience poor generalisation. Feb 25, 2020 at 22:01
• @NuclearWang The splitting process is always done through gini or entropy and therefore always maximises accuracy - the only way you can influence F1 or AUC on a single tree would be through finding the subtree that optimizes those via postpruning, or to choose the hyperparameters of the ensambling procedure. Therefore yes - standard trees do not handle unbalanced classes, but some optimization around can. Feb 26, 2020 at 15:44

Decision trees implementations normally use Gini index or Entropy for finding splits. These are functions that are maximized when the classes in a node are perfectly balanced - and therefore reward splits that move away from this balance. This means that the splits are always done assuming that the classes distribution is $$1/K$$ (or 50-50 in the binary case), and this is particularly clear when classes are VERY unbalanced. In those cases, since classification is done by majority voting, most of the leaf nodes will contain majority class elements and the performance will be sub-optimal.