The equation for lasso and ridge regression are given as follows in the ISLR textbook: enter image description here

enter image description here

The dual form of the above equations are given in terms of budget as below:

enter image description here

I am wondering if there is a relationship between the regularization parameter lambda in the first set of equations and the budget s in the second set of equations. Does increasing lambda mean that budget is less ( or decreasing?) and vice-versa ?

Please correct me if i am wrong. To my understanding, increasing λ decreases the beta coefficients towards zero (and exactly to 0 in Lasso). But, increasing the budget s relaxes the constraints placed on the beta coefficients and therefore, allows larger beta coefficients. To this effect, λ and budget s, have an inverse relationship on the beta coefficient estimates. So, increasing lambda amounts to decreasing the budget s and thus allows for more accurate models that fit the true function closely. Is the above argument correct ?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.