Effect of class imbalance on logistic regression (mathematical basis) [duplicate]

Question

A number of posts, and papers, state that logistic regression (LR) is robust in the face of class imbalance. Unless the imbalance is extreme (e.g., events=0.01 or less), with adequate sample sizes model performance does not degrade. Even in extreme cases, performance may not be affected. I have tested this empirically and found this to be true, at least for events=0.05 or greater. As long as there are a sufficient number of minority cases, LR performance does not typically suffer.

What is the mathematical basis for why LR models perform well in the face of class imbalance? How can this be demonstrated mathematically?

Answer 1

Because the final classifier will still be well-calibrated even when the training data is class-imbalanced:

$$\sum_{i=1}^m p^{(i)} =\sum_{i=1}^m y^{(i)}$$ which means the predictions will be the true probabilities. And for each feature, the sum of the feature values weighted by the labels 1 and 0 will be equal to the feature values weighted by the probabilities.

$$\sum_{i=1}^m h_\theta\left(x^{(i)}\right)x_j^{(i)}=\sum_{i=1}^m y^{(i)}\,x_j^{(i)}$$

For the detailed derivation please refer to my answer here.