There are already posts about warnings from R dealing with logistic regression and linear separation such as this one. I just wanna make sure if in Python Scikit Learn this problem is all solved by the L1/L2 regularization part in the optimization function. In other words, it's safe to say users will not get any infinite MLE estimates warnings from sklearn.linear_model.LogisticRegression?


1 Answer 1


Yes, sklearn.linear_model.LogisticRegression uses penalized logistic regression which "solves" the problem of perfect separation. If you set C to something too large, you might still end up with bad results, though.

  • $\begingroup$ Why we need to bother with regularization when data are linearly separable? I mean the weights can go to infinity but the decision boundary won't change. $\endgroup$
    – ado sar
    Dec 19, 2022 at 13:23
  • $\begingroup$ @adosar Two things here: computational and statistical. Computationally (what this question was about), it's annoying to have your weights go to infinity; small regularization prevents that. Statistically, remember that if the dimension is greater than the number of data points, then almost any dataset (in the measure theoretic sense) linearly separable; this is an example where you might expect regularization to be important for finding the "right" decision boundary to generalize well. (If the dimension is high but less than $n$, it can still be very helpful, depending.) $\endgroup$
    – Danica
    Dec 31, 2022 at 9:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.