Timeline for Coordinate descent soft-thresholding update operator for LASSO
Current License: CC BY-SA 4.0
19 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| S Dec 16, 2022 at 14:40 | history | edited | User1865345 | CC BY-SA 4.0 |
added 6 characters in body
|
| Dec 16, 2022 at 13:41 | review | Suggested edits | |||
| S Dec 16, 2022 at 14:40 | |||||
| Jun 8, 2021 at 12:34 | comment | added | Xavier Bourret Sicotte | @Dennis you would need to read through textbooks on sub-differential theory to get a proper answer to that question, which is way beyond the level of mathematics displayed in this answer. I wouldn't know sorry | |
| Jun 7, 2021 at 9:21 | comment | added | Dennis | @XavierBourretSicotte Thank you for detailed answer. Can you please elaborate on the following: "For the second case we must ensure that the closed interval contains the zero so that $\theta_j=0$ is a global minimum". Why is $0$ a global minimum? How can we set the derivative to $0$ if there's no $\theta_j$ in it? | |
| May 31, 2020 at 21:34 | comment | added | William | Does this work for underdetermined system of equation? | |
| S Mar 14, 2020 at 19:11 | history | suggested | HJ Liang | CC BY-SA 4.0 |
Your answer is very concise and helpful. There are two minor errors, one of which is typo. For the other error, lasso only shrinks the coefficients of predictors except constant term.
|
| Mar 14, 2020 at 13:30 | review | Suggested edits | |||
| S Mar 14, 2020 at 19:11 | |||||
| Jul 3, 2018 at 8:06 | history | edited | Xavier Bourret Sicotte | CC BY-SA 4.0 |
added 98 characters in body
|
| Jun 19, 2018 at 8:41 | history | edited | Xavier Bourret Sicotte | CC BY-SA 4.0 |
added 1 character in body
|
| Jun 15, 2018 at 21:36 | comment | added | rook1996 | Look at stats.stackexchange.com/questions/351631/… for the question regarding the coordinate descent in the lasso case | |
| Jun 14, 2018 at 18:30 | comment | added | rook1996 | Let us continue this discussion in chat. | |
| Jun 14, 2018 at 13:30 | comment | added | rook1996 | I'll open a new threat in a few hours, thanks in advance | |
| Jun 14, 2018 at 13:04 | comment | added | Xavier Bourret Sicotte | Actually, having a look at the various cases it is more complicated than that... the shape of the cost function depends on lambda value, and as a result the path of the coordinate descent also depends on it. In particular it depends on whether the solution occurs at one of the vertices of the l1 diamond or not.. if you ask a new question I can post my findings - its an interesting question | |
| Jun 14, 2018 at 12:01 | comment | added | Xavier Bourret Sicotte | My intuition is that it will be the same staircase gradient descent as for the OLS case, but the cost function will be in the shape of an inverted pyramid. Checkout my code in these blogposts and try to plot it for yourself ? I'd be keen to see the 3D visualization of the coordinate descent blog post | |
| Jun 14, 2018 at 11:53 | comment | added | rook1996 | I need just a geometrical Interpretation in terms of coordinate descent as here en.wikipedia.org/wiki/Coordinate_descent#/media/… for the convex and differentiable case | |
| Jun 14, 2018 at 10:25 | comment | added | Xavier Bourret Sicotte | If you want a feel and intuition around subdifferentials have a look at the blog post I linked or at my own blog post here - For a proof of the stationary condition you'll have to dig into some advanced litterature, here maybe | |
| Jun 14, 2018 at 10:13 | comment | added | rook1996 | Can I ask you one question regarding to that ? Not sure If I should open a new thread. You linked the lecture notes of tibshirani. On slide 6: How can I explain graphicaly, that there is a global minimizer in the case of subdifferentials | |
| Jun 13, 2018 at 16:09 | history | edited | Xavier Bourret Sicotte | CC BY-SA 4.0 |
added 2 characters in body
|
| Jun 13, 2018 at 9:45 | history | answered | Xavier Bourret Sicotte | CC BY-SA 4.0 |
