I try to tune alpha and
lambda.1se (the largest value of lambda such that the error is within 1 standard error of the minimum) for an elastic net. In the
glmnet package it is possible to tune
lambda.1se, but it is not possible to tune alpha and lambda at the same time.
Such a tuning of both parameters is possible with the
caretpackage. However, as a default
caret only provides
lambda.min (the value of lambda that gives the minimum mean cross-validated error). I need
lambda.1se, but unfortunately I can not figure out how to calculate
lambda.1se with the
Question 1: Is there a way to tune
caret or any other package?
If the answer to question 1 is no:
Question 2: Is there a way to calculate
lambda.1se by hand and tune it manually?
Below, you can find some reproducible code, which illustrates the problem:
library("caret") library("glmnet") set.seed(1234) # Some example data N <- 1000 y <- rnorm(N, 5, 10) x1 <- y + rnorm(N, 2, 10) x2 <- y + rnorm(N, - 5, 20) x3 <- y + rnorm(N, 10, 200) x4 <- rnorm(N, 20, 50) x5 <- rnorm(N, - 7, 200) x6 <- rbinom(N, 1, exp(x1) / (exp(x1) + 1)) x7 <- rbinom(N, 1, exp(x2) / (exp(x2) + 1)) x8 <- rbinom(N, 1, exp(x3) / (exp(x3) + 1)) x9 <- rbinom(N, 1, exp(x4) / (exp(x4) + 1)) x10 <- rbinom(N, 1, exp(x5) / (exp(x5) + 1)) data <- data.frame(y, x1, x2, x3, x4, x5, x6, x7, x8, x9, x10) # Tune parameteres with caret and glmnet # Set up grid and cross validation method for train function lambda_grid <- seq(0, 3, 0.1) alpha_grid <- seq(0, 1, 0.1) # Specify conditions for a tuning via caret trnCtrl <- trainControl(method = "repeatedCV", number = 10, repeats = 5) # Create grid for the cross validation srchGrid <- expand.grid(.alpha = alpha_grid, .lambda = lambda_grid) # Cross validation my_train <- train(y ~., data, method = "glmnet", tuneGrid = srchGrid, trControl = trnCtrl) # Best tuning parameters my_train$bestTune # Corresponds to lambda.min; lambda.1se is needed
One way to calculate
lambda.1se could be to use the tuned alpha of
caret within another cross validation based on
glmnet. However, I am not sure if the derived value would be optimal.
# Cross validate with fixed alpha based on glmnet cv <- cv.glmnet(x = as.matrix(data[ , colnames(data) %in% "y" == FALSE]), y = y, alpha = as.numeric(my_train$bestTune), family = "gaussian") cv$lambda.1se # Final value lambda.1se