7
$\begingroup$

I apologize in advance if this question is basic.

I am trying to use LASSO for variable selection, with an implementation in R. I currently have 15 predictors, and looking to reduce the variable space and select the best predictors only, to be included in my final factor model.

Some have advised me to use LASSO for this purpose. However, after reading some documentation about the subject, I am still unsure of how to choose the tuning parameter $\lambda$. As for the implementation in R, I attempted to use the glmnet package:

Get value of tuning parameter $\lambda$

library(glmnet)
ans <- cv.glmnet(data, return[,1], standardize = TRUE)
par(mfrow=c(1, 2))
plot(ans$glmnet.fit, "norm", label=TRUE)
plot(ans$glmnet.fit, "lambda", label=TRUE)
dev.new()
plot(ans)

However, I am not sure how to interpret the results. I then run the following:

model <- cv.glmnet(data, return[,1], standardize = TRUE)
coef(model)

Could anyone please clarify the results of LASSO in R?

Thanks,

$\endgroup$
9
$\begingroup$

By calling cv.glmnet with default arguments you're k-fold cross-validating on lambda with k = 10. The fitted model will use the 1-standard-error-from-min value of lambda by default, and you can get the value by calling cv.glmnet.object$lambda.1se. See page 5 of the vignette: http://cran.r-project.org/web/packages/glmnet/glmnet.pdf.

$\endgroup$
6
  • 2
    $\begingroup$ Thank you for your comment! My question then is: why is the lambda.1se the chosen value for $\lambda$ by default? $\endgroup$
    – Mayou
    Aug 26 '13 at 14:42
  • $\begingroup$ I have also applied what you suggested, and updated my question. Could you please take a look once again? Thanks $\endgroup$
    – Mayou
    Aug 26 '13 at 14:47
  • 1
    $\begingroup$ Most likely because it was proposed in Tibshrani (1996) or in the book Elements of Statistical Learning, also co-authored by him. $\endgroup$
    – tmakino
    Aug 26 '13 at 14:49
  • $\begingroup$ That's correct, it looks like 3 of your coefficients were set to 0. $\endgroup$
    – tmakino
    Aug 26 '13 at 14:50
  • 7
    $\begingroup$ Regarding the use of lambda.1se, here's the comment in the LASSO paper, citing ESLII: "We often use the 'one-standard-error' rule when selecting the best model; this acknowledges the fact that the risk curves are estimated with error, so errs on the side of parsimony (Hastie et al. 2009)." See p.18: jstatsoft.org/v33/i01/paper $\endgroup$
    – David Marx
    Aug 26 '13 at 15:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.