From my understanding lasso more aggressive bring the weights to zero when the weights are less than 1. While, ridge will more aggressive bring weights to 0 (I know that ridge won't actually bring them to zero, but make them smaller). This is because of their penalty. My question if the weights are larger than 1, which regularization technique, lasso or ridge, bring the weights more aggressive to zero. I believe that it is ridge since after the weights are more than 1 ridge the lambda will become exponentially higher. Is my assumption/reasoning correct?

  • $\begingroup$ one way to approach this is to make toy examples of weight vectors (peppered with some $\beta_i >1$ that reflect your actual application) and plug them into the lasso and ridge functions (L1 and L2 norms) separately and see what comes out as $\lambda_1$ and $\lambda_2$ are respectively tuned $\endgroup$
    – develarist
    Sep 18, 2020 at 18:24
  • $\begingroup$ Oh ok! Thanks for the advice. I'll do that. $\endgroup$
    – Kliocontar
    Sep 19, 2020 at 14:23


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.