if two features correlate exactly to the target variable, how does Decision Tree choose between them as the root node to split?

I have a dataset, that consists of three columns, A,B and X.

A and B are the feature columns and X is the target column. They are all numerical values. I am trying to predict which feature contributed to X.

The dataset is as follows

A B X
1 1 0
2 2 1
2 2 1
2 2 1
1 1 0


Features A and B are in direct correlation with our target X. On what basis does the Decision Tree Scikit Learn manage to choose the root node?

I have a case, where it always chooses A as the root node, instead of B when B should be the root node. I am trying to understand how does it do that so I can find a solution for the problem.

• If they are in direct correlation with your target, why is there a preference on which one to choose as the root node? – adrin Apr 4 '18 at 16:06
• @adrin the feature name is what matters – Victor Apr 4 '18 at 16:07
• could you elaborate? it's really not clear why you'd prefer one over the other. – adrin Apr 4 '18 at 16:08

I not sure which Splitter is used in that example, but basically, it finds the first best improvement it can possibly have as a splitter.

The relevant code looks like this:

if current_proxy_improvement > best_proxy_improvement:
best_proxy_improvement = current_proxy_improvement
current.threshold = (Xf[p - 1] + Xf[p]) / 2.0

if current.threshold == Xf[p]:
current.threshold = Xf[p - 1]

best = current  # copy


This means a new one is chosen if it's strictly better than the older improvement. If in the source you change the if current_proxy_improvement > best_proxy_improvement: to if current_proxy_improvement >= best_proxy_improvement:, then it'll choose the last one.