Timeline for Proving Ridge Regression is strictly convex
Current License: CC BY-SA 4.0
14 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Nov 4, 2020 at 15:09 | comment | added | whuber♦ | @Henry Then please take a look at my link or at the answer posted here by Firebug. | |
| Nov 4, 2020 at 15:06 | comment | added | Henry | @whuber - I can accept ridge regression is a least squares method, but I would not see it as ordinary: you are minimising the sum of the squares of the residuals plus the square of something else | |
| Nov 4, 2020 at 14:56 | comment | added | whuber♦ | @Henry and when $\lambda$ otherwise is positive, it is still OLS regression.. In short, this objective function is a squared Euclidean distance to a point, whence it is (obviously) strictly convex. | |
| Nov 4, 2020 at 0:00 | history | tweeted | twitter.com/StackStats/status/1323777001421942784 | ||
| Nov 3, 2020 at 16:18 | history | edited | Firebug |
edited tags
|
|
| Nov 3, 2020 at 16:17 | answer | added | Firebug | timeline score: 2 | |
| Nov 3, 2020 at 15:25 | comment | added | Henry | If $\lambda$ is $0$ then this is ordinary least squares rather than ridge regression | |
| Nov 3, 2020 at 12:35 | history | became hot network question | |||
| Nov 3, 2020 at 5:19 | vote | accept | user8714896 | ||
| Nov 3, 2020 at 5:18 | answer | added | Thomas Lumley | timeline score: 21 | |
| Nov 3, 2020 at 5:00 | comment | added | Sycorax♦ | Since the second partial derivative of $f$ is a matrix, you'll need to adapt your definition to the case of a matrix. You can show that $\frac{\partial ^2 f}{\partial \beta^2}$ is positive semi-definite. What remarks about convexity can you make about p.s.d. matrices? | |
| Nov 3, 2020 at 4:47 | comment | added | user8714896 | @Sycorax I assumed that no entries in the resulting matrix can be negative in order for the function to be strictly convex. Or is that not the case? Is convexity different if it's a matrix? | |
| Nov 3, 2020 at 4:44 | comment | added | Sycorax♦ | $X^\top X$ is a matrix, not scalar. In what sense do you mean "negative"? | |
| Nov 3, 2020 at 4:32 | history | asked | user8714896 | CC BY-SA 4.0 |