4
$\begingroup$

I try to tune alpha and lambda.1se (the largest value of lambda such that the error is within 1 standard error of the minimum) for an elastic net. In the glmnet package it is possible to tune lambda.1se, but it is not possible to tune alpha and lambda at the same time.

Such a tuning of both parameters is possible with the caretpackage. However, as a default caret only provides lambda.min (the value of lambda that gives the minimum mean cross-validated error). I need lambda.1se, but unfortunately I can not figure out how to calculate lambda.1se with the caretpackage.

Question 1: Is there a way to tune lambda.1se via caret or any other package?

If the answer to question 1 is no:

Question 2: Is there a way to calculate lambda.1se by hand and tune it manually?

Below, you can find some reproducible code, which illustrates the problem:

library("caret")
library("glmnet")

set.seed(1234)

# Some example data
N <- 1000
y <- rnorm(N, 5, 10)
x1 <- y + rnorm(N, 2, 10)
x2 <- y + rnorm(N, - 5, 20)
x3 <- y + rnorm(N, 10, 200)
x4 <- rnorm(N, 20, 50)
x5 <- rnorm(N, - 7, 200)
x6 <- rbinom(N, 1, exp(x1) / (exp(x1) + 1))
x7 <- rbinom(N, 1, exp(x2) / (exp(x2) + 1))
x8 <- rbinom(N, 1, exp(x3) / (exp(x3) + 1))
x9 <- rbinom(N, 1, exp(x4) / (exp(x4) + 1))
x10 <- rbinom(N, 1, exp(x5) / (exp(x5) + 1))

data <- data.frame(y, x1, x2, x3, x4, x5, x6, x7, x8, x9, x10)

# Tune parameteres with caret and glmnet

# Set up grid and cross validation method for train function
lambda_grid <- seq(0, 3, 0.1)
alpha_grid <- seq(0, 1, 0.1)

# Specify conditions for a tuning via caret
trnCtrl <- trainControl(method = "repeatedCV",
                    number = 10,
                    repeats = 5)

# Create grid for the cross validation
srchGrid <- expand.grid(.alpha = alpha_grid, .lambda = lambda_grid)

# Cross validation
my_train <- train(y ~., data,
              method = "glmnet",
              tuneGrid = srchGrid,
              trControl = trnCtrl)

# Best tuning parameters
my_train$bestTune # Corresponds to lambda.min; lambda.1se is needed

One way to calculate lambda.1se could be to use the tuned alpha of caret within another cross validation based on glmnet. However, I am not sure if the derived value would be optimal.

# Cross validate with fixed alpha based on glmnet
cv <- cv.glmnet(x = as.matrix(data[ , colnames(data) %in% "y" == FALSE]), 
            y = y, alpha = as.numeric(my_train$bestTune)[1], family = "gaussian")
cv$lambda.1se # Final value lambda.1se
$\endgroup$
4
  • 1
    $\begingroup$ What's the reason for not using glmnet? $\endgroup$
    – mark999
    Mar 26 '17 at 7:39
  • $\begingroup$ Thanks for your comment @ mark999. glmnet can not be used, since it does not provide a tuning of alpha and lambda at the same time. In glmnet, alpha is usually held fix and the tuning is just done for lambda. I edited my question to make it more clear. $\endgroup$ Mar 27 '17 at 6:56
  • $\begingroup$ I am actually facing the same problem here. I wish I could find a specific documentation explaining how caret performs cross validation to tune $\alpha$ and $\lambda$. $\endgroup$ Jul 12 '17 at 18:31
  • $\begingroup$ I think the question is also relevant when someone tries to use packages like grpreg which doesn't give you lambda.1se like in glmnet. But probably more suited for stackoverflow.com $\endgroup$
    – Blain Waan
    Oct 7 '19 at 3:58