I know that input data is not of much use, but I still want to understand how are we going to split the data (in the decision tree) in this case (because all the features should end-up giving the same amount of information gain)

Any thoughts on this?

  • $\begingroup$ It will probably depend on your specific implementation what happens in this edge case, but the most reasonable assumption is that there will simply be no splits at all. $\endgroup$ Mar 8, 2023 at 17:05
  • $\begingroup$ @StephanKolassa - If I use the information gain to split the nodes, then for this case are you saying that since there's no other value for target variable, we don't really have anything to put on the other side of the tree, right? $\endgroup$ Mar 8, 2023 at 17:14
  • 1
    $\begingroup$ Exactly. Splitting gains no information, does not reduce Gini impurity, and so on and so forth: splitting does not help in any way, so any reasonable implementation should not split at all. (But edge cases are typically where the fun bugs are.) $\endgroup$ Mar 8, 2023 at 17:15


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.