I am aware that categorical variables should be one hot encoded before modeling with random Forests. But I am not entirely sure why.

Lets say we have a predictor categorical variable with 7 levels. The tree should be able to find similarities/differences within this variable if it is numerically encoded. Why do we have to one hot encode categorical variables?

How can I simulate a regression to showcase the difference?


The problem is the decision tree construction within the random forest. When constructing a DT one selects for all (numeric) attributes all possible (actually seen) values $x$ of that attribute such that making the decision

$$ value < x $$

will generate the most information gain, for all combinations of attributes and values.

If the attribute is categorical, the decision would have to be

$$ value \in subset $$

But that would require an exponential search for all possible subsets of the attribute, which makes the algorithm much less usable.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.