I have data in which the dependent variable is binary with a highly-skewed distribution: <1% records are 1 (doers), >99% records are 0 (non-doers). I'm using logistic regression to predict the probability that new records are doers.
To handle this rare-event situation, I made multiple samples of non-doers that are size-matched to the number of doers (e.g., sample 1 has the 100 doers and 100 non-doers, sample 2 has the 100 doers and a different set of 100 non-doers, etc.).
If I fit a logistic regression to each sample, how do I make an ensemble model to assign probabilities to new records? Samples have different observations and perform their own feature selection, so they have different feature spaces, which precludes averaging feature coefficients.
Do you have any suggestions for how I can build an ensemble model to take into account the models from all of my samples to compute probabilities?