# Understanding MultiClass Categorical Decision Tree Structure

I have printed the structure of a CART decision tree, from sci-kit learn, but I don’t understand it.

It’s multiclass classification, there are 4 possible labels, and 5 features. There are 5 different values for each feature. This is what the data looks like

Label  Feat1  Feat2  Feat3  Feat4  Feat5
A      A      B      A      C      A
B      B      A      A      B      B
C      A      C      C      A      A
D      A      B      B      D      D


In order to discretize these categorical variables, I have used a LabelEncoder and OneHotEncoder.

This is the result of printing the structure of the Decision Tree. I know that the gini impurity is the decision tree splitting metric, what I really don’t understand is the top of each box, for example [X7]= 0.5 and the value.

I'm not very familiar with sklearn but I'd guess that X[7] <= 0.5 is telling you that the split corresponding to this box is comparing the column of X at index 7 to the value of 0.5, with the left branch being for the cases in which the check returns true. As for the value, i think that's the counts of each class that made it to this split/node. I base that guess on how some of the leaves have value equal to something like [0,0,0,1] and a corresponding gini of 0.

• As the features have been OneHotEncoded, X[7] <= 0.5 does not make sense, – gbhrea Aug 20 '16 at 13:26
• Is it 0 or 1? Then it is true when X[7] == 0 and false otherwise. – jld Aug 20 '16 at 13:51
• OneHotEncoded features are vectors, they look like this [1000,0100,0010,0001] – gbhrea Aug 20 '16 at 14:21
• One hot encoding means you've expanded a categorical variable with k levels into generally k or k-1 binary indicators. You said you have 5 features originally and it looks like you have at least 12 after one hot encoding. Just look at the actual x matrix after one hot encoding and you'll see this. – jld Aug 20 '16 at 14:41