Frank Harrell in this interesting blog post "Classification vs. Prediction" points out that using stratified sampling to handle unbalanced classes is a bad idea, since a classifier trained on an artificially biased data set will then do poorly on real world data set which will be distributed differently from the training data.
He then states that:
Logistic regression on the other hand elegantly handles this situation by either (1) having as predictors the variables that made the prevalence so low, or (2) recalibrating the intercept (only) for another dataset with much higher prevalence.
I'm having a harding digesting this, specifically the idea that logistic regression handles this elegantly:
What does he mean in (1): If a disease is really rare how, would we include that as a feature? Or malicious attacks on a network are very rare compared to legitimate logins, how would that be included as a feature?
In (2): Doesn't recalibrating the intercept in a logistic regression simply amount to playing around with the classification threshold - which can be achieved with all sorts of binary classification methods (and is achieved implicitly by biasing the training data set) ?
Moreover, isn't the bias introduced to the classifier a desirable outcome, given that our purpose is to detect the rare cases (in terms of the precision/recall tradeoff) ?