Timeline for Lasso regression Mathematical intuition
Current License: CC BY-SA 4.0
6 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jul 26, 2022 at 19:52 | comment | added | littleO | We can use the KKT conditions (Lagrange multiplier optimality conditions) from convex optimization to show that a minimizer for problem 2 (which has a hard constraint) is also a minimizer for problem 1, for a certain value of $\lambda$. The parameter $\lambda$ is a Lagrange multiplier. | |
| Jul 21, 2022 at 13:21 | comment | added | Stephan Kolassa | I don't think it's of much interest in itself. If you look at the original Tibshirani (1996) paper, the lasso is actually introduced by an analogue of equation (6.8), and this of course motivates the term "lasso": the coefficients are constrained in a hard way (through their 1-norm), not in a soft way (per 6.7, where coefficient size can be traded off against model fit). I think the (nowadays more common) formulation (6.7) simply came later. | |
| Jul 21, 2022 at 13:10 | comment | added | Dan | Thank you for your reply! I do have one question however. In answer no 1, you said, "for every 𝜆, there is one 𝑠 such that the minimizer 𝛽 of (6.7) for 𝜆 is equal to the minimizer of (6.8)". Why is that particular s value of interest? | |
| Jul 21, 2022 at 12:10 | vote | accept | Dan | ||
| Jul 21, 2022 at 10:07 | history | edited | Stephan Kolassa | CC BY-SA 4.0 |
added 38 characters in body
|
| Jul 21, 2022 at 7:48 | history | answered | Stephan Kolassa | CC BY-SA 4.0 |