# Lasso vs Ridge Regression

My question relates on the Ridge vs Lasso Regression. I know the difference in the cost function (ridge penalizes sum of quadratic coefficients, lasso penalizes sum of absolute value of coefficients). Moreover, I also know that Lasso is able of reducing some coefficients completely to zero while ridge only does towards zero.

So my question is whether one can therefore say from a theoretical perspective that Lasso should have a lower variance (generalizes better) but a higher bias than Ridge because of the above mentioned property of reducing coefficients completely to zero (of course if one applies the same strength of regularization for both of them)?

Thank you.

There are several issues with the way you describe the premise of the question. For one, it is meaningless to say "the same strength of regularization." The fact that you may use the same greek letter for regularization parameters in ridge and lasso doesn't make them directly comparable. Just use $$\alpha$$ in ridge and $$\lambda$$ in lass for regularization paramater to see my point. You can drive coefficients to zero with both methods by cranking up the regularization.
• @alphaH, the parameters belong to different functions with different units of measure even. Suppose, you regressed weight on height of a student, then in $\alpha\times h_i^2$ term $\alpha\sim 1/cm^2$ while in $\lambda|h_t|$ term $\lambda\sim 1/cm$. It's like comparing the "strength" of coefficient of linear and quadratic terms in a polynomial, it makes no sense Oct 11, 2021 at 19:50