7
$\begingroup$

I know in decision tree, we select features which maximize information gain (IG) to split data. My question is that, does such selections need to be the same in the same layer?

Suppose data has features X: sunny = {True, False}, windy = {True, False}, holiday = {True, False} and Y: play = {True, False}.

From root, assume sunny gives maximum IG, such that True and False groups are split in the first child layer. Next, should we consider IG using windy (assume windy gives maximum IG) on both groups to split, or we can allow True group to use windy whereas False group to use holiday to split? The later cases means that we can just consider to maximize IG on local branch.

$\endgroup$
3
  • 1
    $\begingroup$ The second. There's no reason to constrain a tree to split on the same variable at all nodes at a given level. $\endgroup$ Commented Jul 2, 2018 at 5:27
  • $\begingroup$ Thanks! I think the answer is second but cannot confirm. $\endgroup$
    – TripleH
    Commented Jul 2, 2018 at 5:31
  • 1
    $\begingroup$ It definitely is the second. Furthermore, in general, it is possible to split on the same feature multiple times on a path from the root node to a leaf node. In your example this won't happen, as the features take only 2 values. $\endgroup$
    – George
    Commented Jul 2, 2018 at 6:38

2 Answers 2

5
$\begingroup$

The second option.

There's no reason to constrain a tree to split on the same variable at all nodes at a given level.

$\endgroup$
3
$\begingroup$

Actually, there is a reason to do such a thing. Moreover, sometimes both a feature and a threshold are the same for the whole level.

Read about oblivious decision trees used in CatBoost algorithm - https://towardsdatascience.com/introduction-to-gradient-boosting-on-decision-trees-with-catboost-d511a9ccbd14

Such trees predict faster (cause you can avoid using if-statement for each object separately) and they could solve some problems better (XOR problem, as a toy example)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.