Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I am reading Introduction to Statistical Learning and it is said (as in other websites) that Logistic Regression is unstable compared to Linear Discriminant Analysis in well separated cases. Specifically, on p. 138 in the 7th edition:
When the classes are well-separated, the parameter estimates for thelogistic regression model are surprisingly unstable. Linear discriminant analysis does not suffer from this problem.
Can you please provide a simple example why this is so?
Well, if your goal is discrimination, it doesn't really matter if the estimated parameters are unstable---the class assignments could well be stable! Even, if your goal is risk estimation, even if the individual parameter estimates are unstable, the predicted probabilities could be stable. So maybe there is no reason for worry!
There are many similar posts here, so for some more detail see Why does logistic regression become unstable when classes are well-separated? (maybe a duplicate), Does Multicollinearity between categorical variables affect the predictions in logistic regression?, Discriminant analysis vs logistic regression