I know glmnet(x,y) generates $\lambda$ but I am very curious to know the actual formula that is behind this, generating $\lambda$.
$\begingroup$
$\endgroup$
3
2 Answers
$\begingroup$
$\endgroup$
I had this same question and also ran into confusion in the F90 code in the glmnet package. In the end I took some code from the quadrupen package (at the end of quadrupen.R) and modified it for my purposes. I can confirm that the maximum lambda value produces all zero coefficients in glmnet with alpha=1. I'd love to hear better answers to this question or an implementation of the glmnet fortran version in R --- at least to help with teaching and learning.
### from quadrupen
## GENERATE A GRID OF PENALTIES IF NONE HAVE BEEN PROVIDED
get.lambda.l1 <- function(xs,y,nlambda,min.ratio) {
##xs <- as(x, "dgCMatrix")
## currently not robust to missing values in xs or y
ybar <- mean(y,na.rm=TRUE)
xbar <- colMeans(xs,na.rm=TRUE)
x <- list(Xi = xs@i, Xj = xs@p, Xnp = diff(xs@p), Xx = xs@x)
xty <- drop(crossprod(y-ybar,scale(xs,xbar,FALSE)))
lmax <- max(abs(xty))
return(10^seq(log10(lmax), log10(min.ratio*lmax), len=nlambda))
}
$\begingroup$
$\endgroup$
1
From the documentation, it seems that cross-validation is used on a self-generated sequence for lambda.
This results in the lambda.min being the lambda value in the sequence which produces the smallest cvm (mean cross-validated error) and lambda.1se being the largest lambda in the sequence such that error is within 1 standard error of the minimum.
There is some discussion and illustration in section 6 of the JStatSoft article
-
2$\begingroup$ Your answer addresses a different question than the one being asked. The question of interest is how the sequence of lambda is generated by the function
glmnetin theglmnetpackage inR. I have been wondering about this question and checked the code behind theglmnetfunction, but ultimately faced someFortrancode (since theglmnetpackage is not entirely coded inR) which I had trouble with... I was unable to find the answer in the documentation nor in the JStatSoft article. $\endgroup$ Commented Dec 5, 2014 at 10:13
glmnetin the R packageglmnet? $\endgroup$lambda.min.ratio) it says that the largest value of $\lambda$,lambda.maxis the (data derived) entry value (i.e. the smallest value for which all coefficients are zero). Oncelambda.maxis given, obtaining the rest of the $\lambda$ sequence should be pretty simple. The question remains, how islambda.maxcalculated? $\endgroup$