What is the solution when we want to apply the Random Forest function in R to a predictor with more than 53 categories?

> RandomForestPrediction=function(alpha){
+   d = sort(sample(nrow(MPS), nrow(MPS)*alpha))
+   train<-MPS[d,]
+   test<-MPS[-d,]
+   myNtree=1000
+   myMtry=5
+   myImportance=TRUE
+   mod2 = randomForest(factor(m.Decision)~.,data=train,tree=myNtree,mtry=myMtry,importance=myImportance)
+   fitted=predict(mod2,test,type="response")
+   return(table(fitted,test$m.Decision))
+ }
> RandomForestPrediction(0.7)
Error in randomForest.default(m, y, ...) : 
Can not handle categorical predictors with more than 53 categories.
  • $\begingroup$ Do you actually have a categorical variable with more than 53 categories/levels? Maybe a numerical variable was converted into a factor variable? Inspect the dataset using str(MPS) to identify the variable that causes the problem. If you really have a categorical variable with more than 53 levels, you just cant use the randomForest function, I'm afraid. $\endgroup$ Jun 17 '15 at 9:35
  • $\begingroup$ It's exactly the categorical variable, because I'm applying the Random Forest to the data set from the customers. And one predictor is Occupation with 73 levels. I can not take this variable down because it's known as a valuable predictor! $\endgroup$
    – Metariat
    Jun 17 '15 at 9:38
  • $\begingroup$ Another possibility is to recode the 73 levels into binary dummy variables. Or maybe group the 73 occupations into smaller categories. Other than that, it is simply a limitation of the code of randomForest. Maybe there are other implementations of random forests in R that don't have that limitation. $\endgroup$ Jun 17 '15 at 9:55

In this youtube video, Jeremy Howard explains his technique to deal with this problem in , he separates the variables in two sets according the number of observations for each level :

  • Set 1 : levels for $N_{obs}>100$ or ( $25<N_{obs}<100$ + predictive value)
  • Set 2 : all the rest.

I should mention that I'm new to Random Forest and the luck has just made that I looked at this video two days ago. And even if this technique makes sense to me (separate in two sets with different importance) I can't explain the choice of these thresholds (which are obviously a bit arbitrary and dataset dependent), and at what point one can consider that a level has a honorable predictive value.

  • $\begingroup$ How can you actually identify those variables with more than 53 levels? I have many columns and str(dataframe) only gives me back the name of the columns. Thank you! $\endgroup$
    – vog
    Aug 23 at 10:59

Not the answer you're looking for? Browse other questions tagged or ask your own question.