# RandomForest factor with too many levels

I have a model with about 200,000 training observations, where I am regressing, with 4 factors and 2 continuous variables. One of my features has 927 levels, which is causing the R implementation of randomForest to fail (it has a limit of 32 levels for any feature). Unfortunately, I don't see a simple way to avoid using this factor, or to decompose it into a series of continuous variables. Since my predictors are a mix of categorical and continuous, I thought of trees. Can anyone suggest a different implementation (package or language), ML approach, or a better way to pre-process or massage my inputs?

I'm not an R user, but some general comments about random forest

Discrete features can be treated two ways depending on their property

1. if they have a clear ordering, eg year, (bad/med/good) result, then you can just treat them as continuous.
2. if they can't, eg gender,company name, then they should be dummy'd.
• My factors fall into the second category. When I dummy them, I find myself with roughly 2000 variables, and only a few continuous variables. I can run a linear or generalized linear model on this system. Are there other approaches you recommend? – Ed Fine Mar 17 '13 at 16:12

You might try representing that one column differently. You could represent the same data as a sparse dataframe with dummy variables.

Minimum viable code;

example <- as.data.frame(c("A", "A", "B", "F", "C", "G", "C", "D", "E", "F"))
names(example) <- "strcol"

for(level in unique(example$strcol)){ example[paste("dummy", level, sep = "_")] <- ifelse(example$strcol == level,     1, 0)
}


Have you tried cforest in party package? I know it can handle more than 30-40 levels but I am not sure about 900 levels.

http://www.inside-r.org/packages/cran/party/docs/cforest

You can use sklearn (python) and use OneHotEncoding.

OneHotEncoder basically creates one column per each level for a categorical variable (927 in your case). There created columns can take only 0 or 1 as values. This method by definition puts no limit on the number of levels. However, it does not capture the interaction between levels the same way as R does. The reason cforest doesn't allow you to have more than 30 levels is the huge number of permutations (~2^30).

Sklearn aside, I suggest finding a way to reduce your number of levels.