How do decision tree based ensembles like random forest deal with categorical ("factor") predictor variables? My guess would be that indicator variables are created for each factor via a one-hot-encoding (aka 1 of K coding) but I wonder if that is true? The reason I doubt it is that it seems that such a coding for a factor with many levels would of course results in many additional variables and the selection (random) of which predictors to consider for splitting at each node would not actually be picking "variable 1" or not, but instead would be selecting a potential subset of the indicators that resulted from "variable 1".

I was under the impression that Caret in R expanded all factors out using model.matrix in its train function before passing to (any) modeling algorithm including RF however so that made me reconsider.

Does anyone know how these things are treated?

  • $\begingroup$ That answer doesn't speak to the difference when one creates the one-hot-encoding themselves though. If RF is truly respecting the fact that a factor is one thing, if you one-hot prior, that would be lost (I think). $\endgroup$ – B_Miner May 2 '14 at 20:57