6
$\begingroup$

For tree and randomForest packages in R, the number of levels for a factor (as a categorical variable) is capped at 32. An explanation might be that the number of comparisons at each split becomes very high (2^32 approximately). Why does rpart still work with a factor with larger no. of levels?

$\endgroup$
5
  • 2
    $\begingroup$ I don't know the full reason, but CART uses a trick to reduce the number of splits considered. For regression, the levels of a categorical predictor are replaced by mean of the outcome; for binary responses, levels are replaced by the proportion of outcomes in class 1 (see Elements of Statistical Learning book or link for reason). For categorical predictors, there are some approximations. I don't know why randomForest caps this at 32. $\endgroup$ Nov 14, 2016 at 7:21
  • $\begingroup$ Hi Peter, thanks for your help. Are you aware of any detailed documentation for the rpart package (or a research paper, perhaps)? $\endgroup$ Nov 25, 2016 at 6:55
  • 1
    $\begingroup$ Recursive partitioning (CART) requires about n=100,000 to be reliable. Random forests are for tall and thin datasets and often do not perform well when n is not huge and the number of features is large. $\endgroup$ Sep 27, 2021 at 12:31
  • 2
    $\begingroup$ @Frank Harrell:, where does this n=100,000 figure come from? $\endgroup$
    – JTH
    Sep 27, 2021 at 13:01
  • $\begingroup$ Simulations I've done, and bmcmedresmethodol.biomedcentral.com/articles/10.1186/… $\endgroup$ Sep 27, 2021 at 16:15

1 Answer 1

1
$\begingroup$

Partially answered in comments:

I don't know the full reason, but CART uses a trick to reduce the number of splits considered. For regression, the levels of a categorical predictor are replaced by mean of the outcome; for binary responses, levels are replaced by the proportion of outcomes in class 1 (see Elements of Statistical Learning book or link for reason). For categorical predictors, there are some approximations. I don't know why randomForest caps this at 32.

– Peter Calhoun

For some alternative ideas see Random Forest Regression with sparse data in Python

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.