Timeline for Ridge coefficient estimates do not match OLS estimates when $\lambda$ = 0
Current License: CC BY-SA 4.0
7 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Aug 6, 2020 at 23:30 | vote | accept | bob | ||
| Aug 5, 2020 at 12:48 | answer | added | Sextus Empiricus | timeline score: 5 | |
| Aug 5, 2020 at 7:38 | comment | added | Henry |
Your original example has no degrees of freedom and lm1 no residuals: you would get closer results if you tried something like X <- matrix(rnorm(50), ncol = 5, nrow = 10, byrow = TRUE)
|
|
| Aug 4, 2020 at 23:51 | comment | added | EdM |
The ratio of the square of the sums of the non-intercept coefficients between ridge1 and lm1 is suspiciously close to 0.9: sum(coef(ridge1)[2:10])^2/sum(coef(lm1)[2:10])^2 = 0.8995228. I wonder if the function somehow imposes a minimum 10% penalty. One is always welcome to inspect the source code, but it might be simpler just to ask the package authors about that choice.
|
|
| Aug 4, 2020 at 21:19 | comment | added | bob |
Thank you, @EdM. Tried that, but to no avail. I updated the ridge regression line to ridge1 <- glmnet(X_scaled, y, alpha = 0, lambda = c(50, 40, 30, 20, 10, 9, 8, 7, 6, 5, 4, 3, 2.5, 2, 1.5, 1, .9, .8, .7, .6, .4, .3, .2, .1, .05, .01, .001, .0001, 0)), yet coef(ridge1, s = 0) results in the same coefficient values as before (to the fourth decimal).
|
|
| Aug 4, 2020 at 20:59 | comment | added | EdM |
There ia a warning in the manual for glmnet(): "Avoid supplying a single value for lambda .... Supply instead a decreasing sequence of lambda values. glmnet relies on its warms starts for speed, and its often faster to fit a whole path than compute a single fit." See what happens if you instead supply a sequence of $\lambda$ values that ends with 0. If that removes the problem, write that up as an answer here; it's OK to answer your own question. If not, I'm perplexed. Clearly all of the coefficients are slightly penalized.
|
|
| Aug 4, 2020 at 20:49 | history | asked | bob | CC BY-SA 4.0 |