# Does it make sense to have 2 branches resulting in same decision while using one single variable to divide the predictor multiple times?

I have been recently come across a problem that entails using a decision tree that only uses one continuous variable to divide the predictor on multiple threshold, while some splits result in the same decision. The decision tree is as follows:

So I have two questions:

• Does it make sense to use one variable multiple times for dividing the predictor?
• Is it possible that two splits result in the same decision?

I didn't share any codes or data set because I just want to know whether this condition is possible and why we would prefer this.

Does it make sense to use one variable multiple times for dividing the predictor?

Yes, if the variable is useful and has different regions that are not connnected, it can be used mutliple times, e.g. return 1 if 1<x<4 else 0 would require two splits.

Is it possible that two splits result in the same decision?

It is possible as well. Although the final decision is the same, which might have been obtained by taking the majority of cases in each branch, the overall entropy might improve with the branching.

For example, if the distribution is 5-7 for class 0 and 1 respectively, and after the split it becomes (2-3) and (3-4) in the branches, the information gain would be:

h  = -(log(5/12) * (5/12) + log(7/12) * (7/12))
b1 = -(log(2/5) * (2/5) + log(3/5) * 3/5)
b2 = -(log(3/7) * (3/7) + log(4/7) * 4/7)
h - (b1 * 5/12 + b2 * 7/12)


which would result in the following gain

0.000408677

• Thank you very much for your thorough explanation. So do you think pruning also could be a choice here as from a stage on the dividing process won't yield a new result. Commented Dec 16, 2021 at 22:04
• Yes, certainly. Commented Dec 16, 2021 at 22:14
• Could you please explain the cross entropy you used here? I would be very grateful. Commented Dec 17, 2021 at 12:45
• Hi @AnoushiravanR it's the usual one, the entropy: $H(p)=\sum p \log p$. Calculate it for each branch, weighted sum them and subtract it from the entropy calculated in the root. That is the information gain. Commented Dec 17, 2021 at 13:32
• Thank you very much for your comment. Yes I should have noticed. Normally we used Gini index, so this was quite new to me, but very glad that I came to know it now. Commented Dec 17, 2021 at 14:20
1. Absolutely. In one particular case, you may have a non-monotonic relationship between the continuous variable and the target, so that the "correct" tree will have several switches from No to Yes along the leaves from left to right.

2. Yes. While the final "answer" Yes/No can be the same in two leaves from the same parent, their proportions of "yes" samples might be quite different, which may be informative for you. If at the end of the day you only care about the final decision, you can always post-process the tree to simplify the rules.

• Thank you for your answers dear Ben. So do you think pruning here could be a good choice cause from a stage below dividing further does not make any difference. Commented Dec 16, 2021 at 21:53

YES & YES

set.seed(2021)
N <- 10
x0 <- c(runif(N, -4, -2), runif(N, 2, 4))
x1 <- runif(2*N, -2, 2)
plot(x0, rep(1, 2*N), col = 'red')
points(x1, rep(1, 2*N), col = 'blue')


(If someone could run this in R and post the picture, that would be fantastic.)

If you're above $$0$$, the color depends on if you're above $$2$$. If you're below $$0$$, the color depends on if you're below $$-2$$. However, you cannot split at $$2$$ or $$-2$$ and always get the right color. There are reds and blues above $$-2$$. There are reds and blues below $$2$$. Splitting on the same variable multiple times allows us to guess the correct color every time.

In some pseudocode:

if x > 0:
if x < 2:
color = red
if x > 2:
color = blue
if x < 0:
if x < -2:
color = red
if x > -2:
color = blue

• Thank you very much for your code dear Dave. I ran it but I am not sure how I interpret it. Commented Dec 16, 2021 at 21:54
• You can add it via rdrr.io/snippets Commented Dec 16, 2021 at 21:56