Decision Trees - how does split for categorical features happen? A decision tree, while performing recursive binary splitting, selects an independent variable (say $X_j$) and a threshold (say $t$) such that the predictor space is split into regions {$X|X_j < t$} and {$X|X_j >= t$}, and which leads to greatest reduction in cost function.
Now let us suppose that we have a variable with categorical values in {$X$}. Suppose we have label-encoded it and its values are in the range 0 to 9 (10 categories). 


*

*If DT splits a node with the above algorithm and treat those 10 values are true numeric values, will it not lead to wrong/misinterpreted splits? 

*Should it rather perform the split based on == and != for this variable? But then, how will the algorithm know that it is a categorical feature?

*Also, will one-hot encoded values make more sense in this case?

 A: The fundamental question is: what is the nature of the variable?  There are two options:


*

*Variable is numeric, i.e., labeled as 0 to 9 where the magnitude of the number matters. An example is a rating.

*Variable is categorical. i.e., labeled as 0 to 9 where the magnitude of the number does not matter. An example is color (where 0 can stand for green, 1 can stand for blue, etc.)


Based on the wording of your question, I assume the nature of the variable is truly categorical (e.g., a color). With that assumption, here are the answers to your three questions:


*

*The decision tree may yield misinterpreted splits. Let's imagine a split is made at 3.5, then all colors labeled as 0, 1, 2, and 3 will be placed on one side of the tree and all the other colors are placed on the other side of the tree.  This is not desirable.

*In a programming language like R, you can force a variable with numbers to be categorical. If you do that, then you won't run into the issues you are worried about. Here's an example using the mtcars dataset to convert a numeric variable to categorical:


data(mtcars)
mtcars

#carb variable starts out as numeric
typeof(mtcars$carb)

#convert carb variable to categorical
mtcars$carb_categorical <- as.factor(mtcars$carb)
typeof(mtcars$carb_categorical)



*Yes, one-hot encoded values will make more sense.

