Guidance on number of samples per categorical factor/value consolidation in logistic/LASSO Is there any guidance on the "minimum" number of samples per categorical factor when doing logistical modeling (in my case LASSO)?
For example, let's say I have two categorical predictors (classID and state), with many possible values for each:
classID: A, B, C, ...Z
state: AK, AL, AR, ...WY
And each row in the data is a student with a classID and a state, and whether or not the student graduated:


*

*A, MD, TRUE

*A, AK, FALSE

*B, NV, TRUE

*...


But let's say that while there are generally many students per category (e.g. 1000 students from ME, 2000 students in classID B), there are many states and/or classes in which there are only a very few students:
classID F: 4 students
classID J: 3 students
state GA: 10 students
state HI: 4 students
Is there any guidance on when to consolidate or combine such categorical values (e.g. classes F and J are combined into a new classID value classes_with_low_population, and GA and HI are combined into a new state value states_with_low_population)? I assume that having so few students within a particular category must lead to a lot of bias for that particular value (e.g. if all 4 students from HI graduated).
I'm asking this question with logistic modeling/LASSO in mind (would the student graduate), but this could be a more general statistical/modeling question.
I can't seem to find the answer anywhere online (perhaps my vocabulary is not quite correct).
Thanks!
EDIT 1: After writing this question, I almost instantly improved my terminology and found this: https://www.analyticsvidhya.com/blog/2015/11/easy-methods-deal-categorical-variables-predictive-modeling/
While an interesting guide, it seems to be relatively not rigorous. So, if anyone knows of a more rigorous way of consolidating/combining factor levels based on their rarity, I would still very much appreciate it!
EDIT 2: Also:
https://www.quora.com/How-can-we-deal-with-Categorical-Variables-with-Many-Levels-100-efficiently-in-a-regression-What-methods-are-there-to-reduce-the-number-of-levels
 A: As you want the predictors to be related to outcome, when you combine categories you need to make sure that the categories you combine are similarly related to outcome. Thus you shouldn't combine categories just because they happen to have small numbers of cases (like "states with low populations") unless you think that all states with small populations will have similar relations to outcome.
In general, lumping of categories should be based in some way on your knowledge of the underlying subject matter.
For standard logistic regression without penalization, there is a rule of thumb that you should have about 10-20 of the least-frequent-outcome class per predictor that you are including. (For categorical variables, each level except the first counts as a separate predictor.) That would provide guidance on how many categories to lump together to end up with the desired ratio of cases to predictors.
For LASSO you can start with more predictors than that, as the penalization will eventually cut the number of predictors down to near that value. I don't know of any hard-and-fast rule or even a rule of thumb to guide the choice of numbers of cases per predictor level for combining classes. But the penalty against the magnitude of the regression coefficient in LASSO might make it hard for any category that only describes a small number of cases to be included, unless it has a spectacularly large ability to aid in prediction in those few cases. Again, the important issue is to use wisely your knowledge of the subject matter before you start the other aspects of model building.
What might make more sense in this application would be to use quantitative variables that are related to what you presently list as predictive factors. For example, instead of lumping all states with low populations together, if you think that state population is a good predictor it might be better to use "population of the state" itself as a predictor. Then you could also include other characteristics that might be relevant, like per-capita income, state spending on education, education levels of adult population, etc, as additional separate predictors. LASSO then could choose which of those characteristics of the states are most useful for predictions, giving you much more flexibility than a simple state-as-category approach.
