1
$\begingroup$

I am estimating in total three models: Logistic regression without any penalization (as benchmark model), logistic regression with L1 penalization (LASSO) and with L2 penalization (RIDGE). Now i recognize that the coefficients compared to the benchmark model sometimes flip the sign. Is this a possible effect? - Up to now I only thought they could be zero (LASSO) or go towards zero (RIDGE).

Thanks.

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ i don't think anything in regularization prevents or prescribes sign changes in coefficients $\endgroup$
    – Aksakal
    Oct 17, 2021 at 19:56

1 Answer 1

Reset to default
0
$\begingroup$

Both methods will drive your weights towards zero, but they do not differentiate between positive and negative values. Just look at the penalty values for two potential weights, for example -10 and 10:

L2: The penalty is w^2 for a weight w, and (-10)^2 = (10^2) = 100

L1: The penalty is |w| for a weight w, and |-10| = |10| = 10

So both terms will lower the values of your weights, but will not prevent them from being negative.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ You are right. Didn't think about that. Thanks. $\endgroup$
    – alphaH
    Oct 17, 2021 at 20:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.