I'm working with some categorical data that I want to use for prediction with a machine learning algorithm. Many of my my categorical features have multiple categories that contain very few positive observations.
I've seen approaches that merge categories in scenarios such as this in order to create a category with more positive observations (where it makes sense conceptually) .
My question here is whether this is the best approach, and if so whether there is a rule of thumb for how small a category needs to be to justify merging its with another category? I'm also interested to know whether some kind of statistical analysis should be applied to categories before any decision is made to merge them, or whether it is legitimate in most cases to use a priori knowledge and common sense in merging categories!?
A relevant example of one of a category where I'm considering this approach is as follows:
Old Category Size of category (% of all records) Sunny 82% Raining 17% Snowing 0.1% Foggy 0.9% New Category Size of category (% of all records) Good Weather 82% Bad Weather 17%