1
$\begingroup$

I'd like to pick the optimal lambda and alpha using the Glmnet package. I'm open to all models (Ridge, Lasso, Elastic). I'm assuming some out of sample error/cross validation is the best model selection criteria.

Macro <- read.csv("P:/Earnest/Old/R/Input.csv")
x <- Macro[1:13,3:21]
x <- as.matrix(x)

y <- Macro[1:13,2:2]
y <- as.matrix(y)

t <- Macro[14:14,3:21]
t <- as.matrix(t)

Right now, I'm using the following code. The below code presupposes alpha = .5 (elastic), and that lambda.min is the ideal lambda.

fit <-glmnet(x, y, alpha = .5, lambda = NULL) 
cv.fit=cv.glmnet(x,y, alpha = .5, lambda = NULL)
min <- cv.fit$lambda.min
predict(fit ,t, s = min)

Questions: How do I know what is the ideal alpha and lambda? What code can I use to test various lambda/alpha combinations, to find the best out of sample error. Is this the right approach and utilization of Glmnet? What questions am I not considering that should be?

The data: enter image description here

$\endgroup$
  • 1
    $\begingroup$ You can use the mean square error to select the tuning parameter lambda $\endgroup$ – score324 Jul 15 '18 at 0:18
  • 1
    $\begingroup$ The manual published with glmnet in R explains rather clearly how to do this, so reading it would be a good place to begin. $\endgroup$ – whuber Jul 15 '18 at 13:49
1
$\begingroup$

It appears that the default in glmnet is to select lambda from a range of values from min.lambda to max.lambda, then the optimal is selected based on cross validation. The range of values chosen by default is just a linear range (on the log scale) from a the minimum value (like 0, or some value for which we set no features to zero) to the maximum value, (which they set to the smallest value for which the model would set all features to zero).

From the glmnet documentation:

lambda can be provided, but is typically not and the program constructs a sequence. When automatically generated, the λ sequence is determined by lambda.max and lambda.min.ratio. The latter is the ratio of smallest value of the generated λ sequence (say lambda.min) to lambda.max. The program then generates nlambda values linear on the log scale from lambda.max down to lambda.min. lambda.max is not given, but easily computed from the input x and y; it is the smallest value for lambda such that all the coefficients are zero. For alpha=0(ridge) lambda.max would be ∞; hence for this case we pick a value corresponding to a small value for alpha close to zero.)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.