2
$\begingroup$

I try to tune alpha and lambda.1se (the largest value of lambda such that the error is within 1 standard error of the minimum) for an elastic net. In the glmnet package it is possible to tune lambda.1se, but it is not possible to tune alpha and lambda at the same time.

Such a tuning of both parameters is possible with the caretpackage. However, as a default caret only provides lambda.min (the value of lambda that gives the minimum mean cross-validated error). I need lambda.1se, but unfortunately I can not figure out how to calculate lambda.1se with the caretpackage.

Question 1: Is there a way to tune lambda.1se via caret or any other package?

If the answer to question 1 is no:

Question 2: Is there a way to calculate lambda.1se by hand and tune it manually?

Below, you can find some reproducible code, which illustrates the problem:

library("caret")
library("glmnet")

set.seed(1234)

# Some example data
N <- 1000
y <- rnorm(N, 5, 10)
x1 <- y + rnorm(N, 2, 10)
x2 <- y + rnorm(N, - 5, 20)
x3 <- y + rnorm(N, 10, 200)
x4 <- rnorm(N, 20, 50)
x5 <- rnorm(N, - 7, 200)
x6 <- rbinom(N, 1, exp(x1) / (exp(x1) + 1))
x7 <- rbinom(N, 1, exp(x2) / (exp(x2) + 1))
x8 <- rbinom(N, 1, exp(x3) / (exp(x3) + 1))
x9 <- rbinom(N, 1, exp(x4) / (exp(x4) + 1))
x10 <- rbinom(N, 1, exp(x5) / (exp(x5) + 1))

data <- data.frame(y, x1, x2, x3, x4, x5, x6, x7, x8, x9, x10)

# Tune parameteres with caret and glmnet

# Set up grid and cross validation method for train function
lambda_grid <- seq(0, 3, 0.1)
alpha_grid <- seq(0, 1, 0.1)

# Specify conditions for a tuning via caret
trnCtrl <- trainControl(method = "repeatedCV",
                    number = 10,
                    repeats = 5)

# Create grid for the cross validation
srchGrid <- expand.grid(.alpha = alpha_grid, .lambda = lambda_grid)

# Cross validation
my_train <- train(y ~., data,
              method = "glmnet",
              tuneGrid = srchGrid,
              trControl = trnCtrl)

# Best tuning parameters
my_train$bestTune # Corresponds to lambda.min; lambda.1se is needed

One way to calculate lambda.1se could be to use the tuned alpha of caret within another cross validation based on glmnet. However, I am not sure if the derived value would be optimal.

# Cross validate with fixed alpha based on glmnet
cv <- cv.glmnet(x = as.matrix(data[ , colnames(data) %in% "y" == FALSE]), 
            y = y, alpha = as.numeric(my_train$bestTune)[1], family = "gaussian")
cv$lambda.1se # Final value lambda.1se
$\endgroup$

closed as off-topic by Sycorax, Michael Chernick, COOLSerdash, Nick Cox, Peter Flom Mar 26 '17 at 12:30

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question appears to be off-topic because EITHER it is not about statistics, machine learning, data analysis, data mining, or data visualization, OR it focuses on programming, debugging, or performing routine operations within a statistical computing platform. If the latter, you could try the support links we maintain." – Sycorax, Michael Chernick, COOLSerdash, Nick Cox, Peter Flom
If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1
    $\begingroup$ What's the reason for not using glmnet? $\endgroup$ – markseeto Mar 26 '17 at 7:39
  • $\begingroup$ Thanks for your comment @ mark999. glmnet can not be used, since it does not provide a tuning of alpha and lambda at the same time. In glmnet, alpha is usually held fix and the tuning is just done for lambda. I edited my question to make it more clear. $\endgroup$ – JSP Mar 27 '17 at 6:56
  • $\begingroup$ I am actually facing the same problem here. I wish I could find a specific documentation explaining how caret performs cross validation to tune $\alpha$ and $\lambda$. $\endgroup$ – mynameisJEFF Jul 12 '17 at 18:31