A client has classified geographical areas into groups. He has done this in various ways, based on substantive concerns, so there is a 2 group division, a 3 group division and so on. He has also measured various qualities of the areas. He wants to use the qualities to classify the areas into groups.

I tried multinomial logistic regression. For different cases, I got complete or quasicomplete separation, with the associated problems of non-existent maximum likelihood estimates, bizarre confidence limits for odds ratios and so on.

But, can I use that logistic regression as a classifier despite these problems?

If not, would a classification tree be better?


1 Answer 1


Yes, you can. The predictive accuracy will probably be bad, because separation usually arises from not having enough data for your predictors. Regularization (as with a ridge or lasso penalty) could help a lot.

Keep in mind that logistic regression is designed more for probabilistic classification (predicting the probability of each case belonging to each class) than plain old classification. So if you don't care about predicted probabilities, you may be better served with something else, like a classification tree.

That said, complete separation is an inherent feature of the data and thus is, in general, just as much of a problem for models other than logistic regression. What you want is regularization or feature selection, which are standard parts of some models.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.