I have a dataset, that consists of three columns, A,B and X.

A and B are the feature columns and X is the target column. They are all numerical values. I am trying to predict which feature contributed to X.

The dataset is as follows

1 1 0
2 2 1
2 2 1
2 2 1
1 1 0

Features A and B are in direct correlation with our target X. On what basis does the Decision Tree Scikit Learn manage to choose the root node?

I have a case, where it always chooses A as the root node, instead of B when B should be the root node. I am trying to understand how does it do that so I can find a solution for the problem.

  • $\begingroup$ If they are in direct correlation with your target, why is there a preference on which one to choose as the root node? $\endgroup$ – adrin Apr 4 '18 at 16:06
  • $\begingroup$ @adrin the feature name is what matters $\endgroup$ – Victor Apr 4 '18 at 16:07
  • $\begingroup$ could you elaborate? it's really not clear why you'd prefer one over the other. $\endgroup$ – adrin Apr 4 '18 at 16:08

I not sure which Splitter is used in that example, but basically, it finds the first best improvement it can possibly have as a splitter.

The relevant code looks like this:

if current_proxy_improvement > best_proxy_improvement:
    best_proxy_improvement = current_proxy_improvement
    current.threshold = (Xf[p - 1] + Xf[p]) / 2.0

    if current.threshold == Xf[p]:
        current.threshold = Xf[p - 1]

    best = current  # copy

This means a new one is chosen if it's strictly better than the older improvement. If in the source you change the if current_proxy_improvement > best_proxy_improvement: to if current_proxy_improvement >= best_proxy_improvement:, then it'll choose the last one.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.