2
$\begingroup$

I have printed the structure of a CART decision tree, from sci-kit learn, but I don’t understand it.

It’s multiclass classification, there are 4 possible labels, and 5 features. There are 5 different values for each feature. This is what the data looks like

Label  Feat1  Feat2  Feat3  Feat4  Feat5
 A      A      B      A      C      A
 B      B      A      A      B      B
 C      A      C      C      A      A
 D      A      B      B      D      D

In order to discretize these categorical variables, I have used a LabelEncoder and OneHotEncoder.

This is the result of printing the structure of the Decision Tree. I know that the gini impurity is the decision tree splitting metric, what I really don’t understand is the top of each box, for example [X7]= 0.5 and the value.

Decision Tree Structure

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

I'm not very familiar with sklearn but I'd guess that X[7] <= 0.5 is telling you that the split corresponding to this box is comparing the column of X at index 7 to the value of 0.5, with the left branch being for the cases in which the check returns true. As for the value, i think that's the counts of each class that made it to this split/node. I base that guess on how some of the leaves have value equal to something like [0,0,0,1] and a corresponding gini of 0.

Improve this answer
$\endgroup$
4
  • $\begingroup$ As the features have been OneHotEncoded, X[7] <= 0.5 does not make sense, $\endgroup$
    – gbhrea
    Commented Aug 20, 2016 at 13:26
  • 1
    $\begingroup$ Is it 0 or 1? Then it is true when X[7] == 0 and false otherwise. $\endgroup$
    – jld
    Commented Aug 20, 2016 at 13:51
  • $\begingroup$ OneHotEncoded features are vectors, they look like this [1000,0100,0010,0001] $\endgroup$
    – gbhrea
    Commented Aug 20, 2016 at 14:21
  • 1
    $\begingroup$ One hot encoding means you've expanded a categorical variable with k levels into generally k or k-1 binary indicators. You said you have 5 features originally and it looks like you have at least 12 after one hot encoding. Just look at the actual x matrix after one hot encoding and you'll see this. $\endgroup$
    – jld
    Commented Aug 20, 2016 at 14:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.