I have a dataset of about n = 100,000 observations and p = 247 predictors with one binomial dependent variable (values are 0, 1)

I run the following code in R:

cvfit.retro = cv.glmnet(retro.x, y=as.factor(retro.y), family='binomial')

where retro.x is the matrix of 247 predictors retro.y is the vector of the binomial variable

The idea behind running the code was to reduce the number of variables to a more manageable number for further analysis for their predictive power in relation to the binomial dependent variable.

However, the cvfit.retro output at a relatively high lambda of exp(-6),

coef(cvfit.retro, s=exp(-6))

shows highly correlated variables being INCLUDED in the model. For example, there are two variables called v708 and v709 that have non-zero coefficients in the CV.GLMNET output and these two variables have a correlation of 0.9995 using the cor() function. These variables are obviously powerful predictors in the model but one of them should have been zeroed out in the LASSO in CV.GLMNET given that alpha defaults to 1 (LASSO) in cv.glmnet

Am I missing something here? Can someone explain how I can fix this issue? Obviously, I could just eliminate one of the correlated variables manually but that kind of partially defeats the purpose of LASSO. Any advice is appreciated. Thank you!


migrated from stackoverflow.com Dec 8 '15 at 9:30

This question came from our site for professional and enthusiast programmers.

  • $\begingroup$ Hmm, never seen this, weird. Lasso is the alpha=1.0 case. Try other values of alpha between 0.05..0.95 (general elastic-net) and see if that makes this go away. If not, can you cook up a reproducible example? $\endgroup$ – smci Feb 24 '17 at 2:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy