Timeline for Why L1 norm for sparse models
Current License: CC BY-SA 4.0
14 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Apr 20 at 3:29 | history | edited | Zhanxiong | CC BY-SA 4.0 |
added 107 characters in body
|
| Apr 19 at 21:36 | history | edited | Zhanxiong | CC BY-SA 4.0 |
added 124 characters in body
|
| Apr 19 at 20:58 | history | edited | Zhanxiong | CC BY-SA 4.0 |
added 2018 characters in body
|
| Jan 24, 2022 at 18:55 | comment | added | Poik | I wonder if there is a proof. It should be semi-trivial to write up. I'll try after work today. | |
| Jan 24, 2022 at 18:52 | comment | added | Poik | You all aren't wrong, but you're missing the point a little. L2 nearly guarantees not touching a minima on an axis, due to the roundness of both constraints here. L1 has more chance of touching on a sparse solution, but it isn't guaranteed, The only one that would have guarantees at finding optimal sparsity is L0, which is a mentioned to be a pain to work with. L0.5 has a higher probability since of the L0.5 ball jut out along the axis. If the intersection is not on an axis, then by the definition of the problem statement, it is not optimal to chose that sparse solution. | |
| Jun 10, 2021 at 13:22 | comment | added | jds | I agree with @wabbit. Why should the contours intersect at the corner of the pyramid constraint region? I could easily re-draw the figure such that the contours touched the edge of this pyramid. | |
| Dec 20, 2020 at 22:47 | comment | added | JP Zhang | Is there any proof? Why can't this figure simply be an artifact? | |
| Mar 25, 2020 at 3:52 | comment | added | dksahuji | It comes from property of Lagrange multipiers. At optima the tangent to the loss and constraint should be shared. Because at non differtiable point you have infinite tangents it's more likely to have those as solutions. | |
| Dec 16, 2017 at 15:50 | comment | added | Tautvydas | Note that with L1 edges are only preferred when $\hat{\beta}$ has different variances over $\beta_1$ and $\beta_2$ axis. In other words when redline distribution is not symmetrical on diagonal $\beta_1 = \beta_2$ axis. If it is symmetrical then the whole edge has the same distance/value/cost. | |
| Dec 3, 2017 at 17:18 | comment | added | kjetil b halvorsen♦ | All the contours will have the same form ... | |
| Dec 3, 2017 at 17:03 | comment | added | Zhanxiong | @HrishikeshGanu Eventually got some time to edit the post. | |
| Dec 3, 2017 at 17:02 | history | edited | Zhanxiong | CC BY-SA 3.0 |
As request, supplement more intuitions on this picture.
|
| Oct 19, 2016 at 16:21 | comment | added | wabbit | The illustration is not very convincing without additional information. E.g. why should the contours of the error be located where they are in the figure? | |
| Apr 4, 2016 at 15:46 | history | answered | Zhanxiong | CC BY-SA 3.0 |