2
$\begingroup$

I am reading Introduction to Statistical Learning and it is said (as in other websites) that Logistic Regression is unstable compared to Linear Discriminant Analysis in well separated cases. Specifically, on p. 138 in the 7th edition:

When the classes are well-separated, the parameter estimates for thelogistic regression model are surprisingly unstable. Linear discriminant analysis does not suffer from this problem.

Can you please provide a simple example why this is so?

$\endgroup$

1 Answer 1

2
$\begingroup$

Well, if your goal is discrimination, it doesn't really matter if the estimated parameters are unstable---the class assignments could well be stable! Even, if your goal is risk estimation, even if the individual parameter estimates are unstable, the predicted probabilities could be stable. So maybe there is no reason for worry!

There are many similar posts here, so for some more detail see Why does logistic regression become unstable when classes are well-separated? (maybe a duplicate), Does Multicollinearity between categorical variables affect the predictions in logistic regression?, Discriminant analysis vs logistic regression

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.