Is it possible for ridge regression in some rare case to reduce some co-efficients to zero(Lambda being huge but less than infinity). Or is it never possible?

Same with LASSO, is it possible for LASSO to reduce all co-efficients to 0 only when lambda is infinity?

If there are examples for these cases it would be helpful.

  • 2
    $\begingroup$ If $X$ is 0, always :-) $\endgroup$ – Benoit Sanchez Feb 14 '18 at 11:44
  • $\begingroup$ Oh right, but can the RSS contour never meet the shrinkage circle on one of the axes when X is not zero? Literature only says it's probability is less, so.. $\endgroup$ – Narahari B M Feb 14 '18 at 11:51
  • 2
    $\begingroup$ If one of X variables has exactly 0 correlation to all other X variables and also to the Y variable, then its coefficient is going to be 0 for any value of ridge penalty. Of course for real data this has probability 0. $\endgroup$ – amoeba Feb 14 '18 at 11:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.