For tree and randomForest packages in R, the number of levels for a factor (as a categorical variable) is capped at 32. An explanation might be that the number of comparisons at each split becomes very high (2^32 approximately). Why does rpart still work with a factor with larger no. of levels?
$\begingroup$
$\endgroup$
5
-
2$\begingroup$ I don't know the full reason, but CART uses a trick to reduce the number of splits considered. For regression, the levels of a categorical predictor are replaced by mean of the outcome; for binary responses, levels are replaced by the proportion of outcomes in class 1 (see Elements of Statistical Learning book or link for reason). For categorical predictors, there are some approximations. I don't know why randomForest caps this at 32. $\endgroup$– Peter CalhounCommented Nov 14, 2016 at 7:21
-
$\begingroup$ Hi Peter, thanks for your help. Are you aware of any detailed documentation for the rpart package (or a research paper, perhaps)? $\endgroup$– Pradnyesh JoshiCommented Nov 25, 2016 at 6:55
-
1$\begingroup$ Recursive partitioning (CART) requires about n=100,000 to be reliable. Random forests are for tall and thin datasets and often do not perform well when n is not huge and the number of features is large. $\endgroup$– Frank HarrellCommented Sep 27, 2021 at 12:31
-
2$\begingroup$ @Frank Harrell:, where does this n=100,000 figure come from? $\endgroup$– JTHCommented Sep 27, 2021 at 13:01
-
$\begingroup$ Simulations I've done, and bmcmedresmethodol.biomedcentral.com/articles/10.1186/… $\endgroup$– Frank HarrellCommented Sep 27, 2021 at 16:15
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Partially answered in comments:
I don't know the full reason, but CART uses a trick to reduce the number of splits considered. For regression, the levels of a categorical predictor are replaced by mean of the outcome; for binary responses, levels are replaced by the proportion of outcomes in class 1 (see Elements of Statistical Learning book or link for reason). For categorical predictors, there are some approximations. I don't know why randomForest caps this at 32.
– Peter Calhoun
For some alternative ideas see Random Forest Regression with sparse data in Python
answered Aug 9, 2018 at 17:15