Suppose the event of interest occurs in approximately $10 \%$ of the cases where the number of cases is around $5,000$. Should you use a penalized logistic regression for this or is regular logistic regression okay? In other words, what qualifies something as a rare event?
Don't do anything special.
However, and this is crucial: choose a good quality measure. And that is not classification accuracy, sensitivity, specificity or similar measures, such as ROC curves. These can be very misleading in the case of unbalanced data, "identifying" that simply labeling everything as the majority class is "optimal". Which it isn't.
Oversampling the minority class or undersampling the majority class won't solve this problem, because it amounts to biasing your model and pretending that the population is different than it truly is. Neither will collecting more data solve your problem, since the relation between majority and minority classes won't change.
Instead, use probabilistic models instead of hard thresholded 0-1 classification, and then use proper scoring-rules. ("Proper" is really part of the term. There are proper and non-proper scoring rules. Classification accuracy is a non-proper scoring rule, and that is why it is not useful.)
Frank Harrell, who knows what he is talking about, has written extensively on the topic:
What you are asking here is basically food for numerous posts, opinions and a lot of research on the field of imbalanced classes. To answer your main question, there is not a straight answer of what a “rare” event is. I personally would say that your case is basically in the boundary. I will list some approaches here but seriously this will be the tip of the iceberg you will have to go back to google and look for more info:
- Do nothing. Sometimes it’s fine to just model the data as it is. Especially for an algorithm like logistic regression which not a classifier and predicts probabilities, these class imbalances will be taken into account.
In the case you want to use the model as a classifier you might consider to:
- Undersample the majority class. Sample observations so that you make the two classes balanced
- Oversample the minority class. Similarly sample some observations from the minority class, obviously with replacement
That being said, I believe that these 3 techniques can get you started. Given that you want prediction accuracy, I would suggest to look into other techniques as well. If you really want to use logistic regression then what I would do is:
- Split the data into train and test sets
- Define a loss function
- Check which approach minimises the loss function in the test set