I try to figure out how a logistic regression is handling data with a 0/1 dependent variable. 95% of the data are ones. The purpose of the model is to estimate the probability to be in one of the groups. Of course there are several explanatory variables included.

Fitting the model leads to unsatisfactory results, since the model will predict for almost every observation that it is in category=1. I tried some very naive weighting, which already improved the model. I simply changed the proportion and reran the fitting process with different weights. The weights changed the proportion in the dependent variable (I have no assumptions about the explanatory variables, so every observation in category=1 gets the same weight) to 50:50, 60:40, 70:30 and 80:20. Trusting some validation measures like AIC, different pseudo R2s and the classification table the model with 50:50 weighting is the best. (The calculations of these measures are mostly based on the log likelihood of the corresponding model. So they will be based on the weighted samples.)

These measures probably won't do the job and maybe are inappropriate to find an optimal weighting procedure. My literature somehow skips this point, although I don't think that very unequal groups in the dependent variable are such an uncommon situation. I heard from a colleague of mine, that I should use an ROC curve to find optimal weights.

Can anybody refer to a book ore solution for this problem?

Or can give hints how to handle it?

  • $\begingroup$ Does this similar thread answer your question? $\endgroup$ – whuber Jan 12 '12 at 17:48
  • $\begingroup$ Just want to clarify, when you measure model performance such AIC, R2, etc, you use weighted sample or the original (ie unweighted) sample? $\endgroup$ – FMZ Jan 13 '12 at 2:02
  • $\begingroup$ The "similar thread" is enough direction. Thanks! I edited the question concerning the GoF measures. $\endgroup$ – Sebastian Jan 16 '12 at 18:21