I am sure most of us have had this problem at some point.
Take for example the random forest algorithm and its implementation in R. This algorithm can not handle a categorical variable with more than (approximately) 55 categories. The work-around for this problem usually involves manually selecting categories with low counts and combining them into a single category. However, users usually have to decide themselves what should be considered as a "low count" as well as the desired count of the combined category. Different selections of these thresholds can result in different models with different performances.
I was wondering if there was a more mathematical approach to this problem. For example - is there any statistical algorithm that might try to randomly decide to combine different categories and decide these thresholds, and then decide which threshold combination to select based on some metric (e.g. best performance (e.g. F-Score, AUC) of the model based on these combination/thresholds)?
Or is it better to just use some algorithm in the background such as the Genetic Algorithm or Simulated Annealing?