In R, does randomForest use terms() of model.matrix?

Question

I am using the randomForest package in R with categorical co-variates. The documentation advises against the formula interface for large data. Following this advice, I prepare the outcome variable and a model.matrix object, M.

My question is does the algorithm treat the columns of M independently, so the mtry variables chosen may be biased to selecting from a factor with many levels? Or is the original factor (available via terms(M)) used when selecting the mtry variables?

I am familiar with cforest and the papers by Strobl et al. that address the bias in randomForest based on the domain of the variables. My question is does the choice of arguments (formula vs matrix) in randomForest exacerbate this bias.

thanks.

Answer 1

tl;dr

randomForest will include and exclude dummy-encoded columns independently if using a model matrix, and together if using a data.frame and formula. So yes, the treatment is different (better?) using the formula interface.

Evidence

randomForest under the hood doesn't break categorical variables into multiple columns. It encodes them as one numeric column with their factor code (using data.matrix). It uses ncat/cat internally to keep track of which variables are numeric and categorical.

For matrix input, this just gets set to 'numeric' for every variable. mtry then causes the sampling of each column with equal probability.