0
$\begingroup$

We use regularized Linear Regression to prevent the model from overfitting (reduce model complexity).

Does the same idea hold with regularized Logistic Regression?

Is regularized Logistic Regression a solution to the problem of separation? if yes, how?

I am sure I had some misunderstanding, can anyone help to clarify that for me.

$\endgroup$
8
$\begingroup$

We use regularized Linear Regression to prevent the model from overfitting (reduce model complexity). Does the same idea hold with regularized Logistic Regression?

Yes. The bias-variance trade-off exists in all areas of statistics.

Is regularized Logistic Regression a solution to the problem of separation?

Yes; even a small penalty on the coefficients will bound them away from infinity. This is because you will not be able to improve model fit to be arbitrarily good without eventually trading-off with an increase in penalty on coefficients.

See also: How to deal with perfect separation in logistic regression?

$\endgroup$
7
  • $\begingroup$ how does the regularized Logistic Regression solve the problem of separation? $\endgroup$ – jeza May 7 '19 at 3:27
  • $\begingroup$ What does it mean to be "solved"? What part of my explanation is unclear? $\endgroup$ – Sycorax May 7 '19 at 3:37
  • $\begingroup$ you said yes regularized Logistic Regression is a solution to the problem of separation, how does this work, how this regularized regression solves this problem. $\endgroup$ – jeza May 7 '19 at 3:39
  • $\begingroup$ Did you read the next sentence? $\endgroup$ – Sycorax May 7 '19 at 3:51
  • $\begingroup$ yes I did read it.. $\endgroup$ – jeza May 7 '19 at 3:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.