Timeline for Maximum penalty for ridge regression
Current License: CC BY-SA 4.0
12 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Oct 8, 2021 at 14:18 | comment | added | Richard Hardy | @TomWenseleers, thank you! | |
| Oct 8, 2021 at 14:06 | comment | added | Tom Wenseleers | @RichardHardy It's been on the back burner for a while, but I will have some bioinformatics students work on this this semester to finish it all off. This was a really old version, but it still needs a lot of work (bug fixed but also implementing more efficient solvers and implementing additional functionality etc): github.com/tomwenseleers/L0glm. There is also the l0ara package, which you could look into (that one is already on CRAN): github.com/wguo1017/l0ara | |
| Oct 7, 2021 at 11:51 | comment | added | Richard Hardy | @TomWenseleers, how are the R package and the update doing? Had forgotten about this thread but found it anew, read the comments again and hence am posting this. | |
| Aug 2, 2020 at 18:12 | comment | added | Ryan Burn | "LOOCV in turn is approximated by generalized cross validation (GCV), but LOOCV should always be better than GCV." -- That's not quite right. GCV is a LOOCV of a rotation of the original regression problem. $\bf{X'} = \bf{Q} \bf{X}$, $\bf{y'} = \bf{Q} \bf{y}$ where $\bf{Q} \bf{Q}^H = \bf{I}$ and the rotation matrix is chosen so as to spread out variance. It avoids certain problem cases that LOOCV has and in many cases is better than LOOCV. For more details and examples where GCV is better, see this blog post and Golub and Wahba. | |
| Jun 28, 2019 at 7:26 | comment | added | Tom Wenseleers | Problem is that I would have to post a thread with the answer already included (as I cannot see any benefits of using plain ridge over adaptive ridge or L0 penalized regression). But I'm working on an R package that fits ridge, adaptive ridge or L0 penalized GLMs with or without positivity constraints on the fitted coefficients and with inference also included. That should be done by next week. Once that's on github I'll use that to update my answer here to give a more targeted answer to your question... Hope that's OK... Just hold on a bit... | |
| Jun 27, 2019 at 11:50 | comment | added | Richard Hardy | An alternative: you could post this as a separate thread and then post a comment here with a link to it. | |
| Jun 27, 2019 at 11:34 | comment | added | Tom Wenseleers | Yes I know what I wrote doesn't exactly answer your question - but I thought it might still be useful to have it here for reference, and it was just too long for a comment... | |
| Jun 27, 2019 at 10:54 | comment | added | Richard Hardy | Better now, thank you. I still wonder whether the answer fits the question. An alternative could be to post this answer on a new thread dedicated to the question of when one should choose iterated adaptive ridge regression instead of ridge regression. I would be happy to upvote elements of such a thread. Meanwhile, for those who are going to choose ridge regression, an answer to the precise question I have posed here might still be of interest. | |
| Jun 27, 2019 at 9:59 | history | edited | Tom Wenseleers | CC BY-SA 4.0 |
added 288 characters in body
|
| Jun 27, 2019 at 9:54 | comment | added | Tom Wenseleers | Well in a way it's maybe not answering your question - rather I would argue not to use ridge but iterated adaptive ridge instead, as you then don't have to tune the regularization parameter at all - rather you can just set it a priori to conform to either maximizing AIC (if you set lambda=2) or BIC (if you set lambda=log(n)). Makes sense? | |
| Jun 27, 2019 at 9:51 | comment | added | Richard Hardy | Thank you for your extensive answer! Could you make it more obvious which part of it addresses my question? | |
| Jun 27, 2019 at 9:41 | history | answered | Tom Wenseleers | CC BY-SA 4.0 |