2
$\begingroup$

I have printed the structure of a CART decision tree, from sci-kit learn, but I don’t understand it.

It’s multiclass classification, there are 4 possible labels, and 5 features. There are 5 different values for each feature. This is what the data looks like

Label  Feat1  Feat2  Feat3  Feat4  Feat5
 A      A      B      A      C      A
 B      B      A      A      B      B
 C      A      C      C      A      A
 D      A      B      B      D      D

In order to discretize these categorical variables, I have used a LabelEncoder and OneHotEncoder.

This is the result of printing the structure of the Decision Tree. I know that the gini impurity is the decision tree splitting metric, what I really don’t understand is the top of each box, for example [X7]= 0.5 and the value.

Decision Tree Structure

$\endgroup$
1
$\begingroup$

I'm not very familiar with sklearn but I'd guess that X[7] <= 0.5 is telling you that the split corresponding to this box is comparing the column of X at index 7 to the value of 0.5, with the left branch being for the cases in which the check returns true. As for the value, i think that's the counts of each class that made it to this split/node. I base that guess on how some of the leaves have value equal to something like [0,0,0,1] and a corresponding gini of 0.

$\endgroup$
  • $\begingroup$ As the features have been OneHotEncoded, X[7] <= 0.5 does not make sense, $\endgroup$ – gbhrea Aug 20 '16 at 13:26
  • $\begingroup$ Is it 0 or 1? Then it is true when X[7] == 0 and false otherwise. $\endgroup$ – jld Aug 20 '16 at 13:51
  • $\begingroup$ OneHotEncoded features are vectors, they look like this [1000,0100,0010,0001] $\endgroup$ – gbhrea Aug 20 '16 at 14:21
  • $\begingroup$ One hot encoding means you've expanded a categorical variable with k levels into generally k or k-1 binary indicators. You said you have 5 features originally and it looks like you have at least 12 after one hot encoding. Just look at the actual x matrix after one hot encoding and you'll see this. $\endgroup$ – jld Aug 20 '16 at 14:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.