I'm working on a model that requires me to look for predictors for a rare event (less than 0.5% of the total of my observations). My total sample is a significant part of the total population (50,000 cases). My final objective is to obtain comparable probability values for all the non-events, without the bias of the groups difference in the logistic regression.
I've been reading the info in the following link:
It advises me first to use a sample of my original sample, containing all the events (1) and a random sample of 1-5 times bigger of the non-event (0) sample.
Then it suggests using weights based on the proportion of the sample 1s to 0s. In the section 4.2 of the linked text, he offers a "easy to implement" weighted log-likelihood that can be implemented in any logit function.
I wish to implement these weights somehow with R's glm(...,family=binomial(link="logit")) or similar function ( the "weights" parameter is not for frequency weighting), but I don't really know how to apply this weighting.
Does anybody knows how to make it or any other alternative suggestion?
Edit1: As suggested bellow, is Firth's method for bias-correction by penalizing the likelihood in the
logistf package a correct approach in this case? I'm not much knowledgeable in statistics, and, while I understand the input and the coefficients/output of the logistic model, what happens in between is still quite a mystery to me, sorry.