# Decision Trees - how does split for categorical features happen?

A decision tree, while performing recursive binary splitting, selects an independent variable (say $$X_j$$) and a threshold (say $$t$$) such that the predictor space is split into regions {$$X|X_j < t$$} and {$$X|X_j >= t$$}, and which leads to greatest reduction in cost function.

Now let us suppose that we have a variable with categorical values in {$$X$$}. Suppose we have label-encoded it and its values are in the range 0 to 9 (10 categories).

1. If DT splits a node with the above algorithm and treat those 10 values are true numeric values, will it not lead to wrong/misinterpreted splits?
2. Should it rather perform the split based on == and != for this variable? But then, how will the algorithm know that it is a categorical feature?
3. Also, will one-hot encoded values make more sense in this case?

The fundamental question is: what is the nature of the variable? There are two options:

1. Variable is numeric, i.e., labeled as 0 to 9 where the magnitude of the number matters. An example is a rating.
2. Variable is categorical. i.e., labeled as 0 to 9 where the magnitude of the number does not matter. An example is color (where 0 can stand for green, 1 can stand for blue, etc.)

Based on the wording of your question, I assume the nature of the variable is truly categorical (e.g., a color). With that assumption, here are the answers to your three questions:

1. The decision tree may yield misinterpreted splits. Let's imagine a split is made at 3.5, then all colors labeled as 0, 1, 2, and 3 will be placed on one side of the tree and all the other colors are placed on the other side of the tree. This is not desirable.
2. In a programming language like R, you can force a variable with numbers to be categorical. If you do that, then you won't run into the issues you are worried about. Here's an example using the mtcars dataset to convert a numeric variable to categorical:
data(mtcars)
mtcars

#carb variable starts out as numeric
typeof(mtcars$carb) #convert carb variable to categorical mtcars$$carb_categorical <- as.factor(mtcars$$carb) typeof(mtcars$carb_categorical)

1. Yes, one-hot encoded values will make more sense.
• Thank you for the detailed response! With respect to #2, does R implementation of DT check the values of categorical variables for equality rather than less and greater than values? (I have no experience in R, hence asking). Aug 9 '19 at 7:40
• It does; the decision tree will treat the categorical variable as one-hot encoded values implicitly. Note that in R, factor variables are limited to a maximum number of levels.
– mnmn
Aug 9 '19 at 14:30