0
$\begingroup$

If I have an imbalanced response variable 80% majority, 20% minority, and my decision tree is not finding any splits. Could this potentially be because of the imbalance in my response?

$\endgroup$
3
  • $\begingroup$ What do you mean by “not finding any splits”? $\endgroup$
    – Tim
    Commented Oct 21, 2022 at 6:34
  • 2
    $\begingroup$ 80:20 is not extreme imbalance. You might have chosen the hyperparameters unwisely, or no feature is informative for discriminating between the classes, or you have a bug somewhere. It would help to provide details. $\endgroup$
    – dipetkov
    Commented Oct 21, 2022 at 6:37
  • $\begingroup$ Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking. $\endgroup$
    – Community Bot
    Commented Oct 21, 2022 at 6:38

1 Answer 1

0
$\begingroup$

Yes, I suspect that is the case, but it could also be true for datasets without an imbalance, but it seems to me to be more likely with an imbalance.

If you are using an algorithm such as CHAID, it will perform a statistical significance test for each split and if the reduction in the cost function does not reach statistical significance. If you are using an algorithm with reduced error pruning, it may initially find splits, only to have them pruned back because they don't reduce the error on a validation set.

In both cases, if there is an imbalance, weakly informative features are likely to split out only a small number of positive examples in the training data, which makes the split less likely to be statistically significant, and more likely to be over-fitting the noise in the dataset and not generalise (and hence be pruned back).

However, this problem will go away (as will most estimation problems) if you add more and more data (with the same class ratio), so it is not the imbalance per se that is causing the problem, but having too few minority class examples.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.