I have a dataset of about n = 100,000 observations and p = 247 predictors with one binomial dependent variable (values are 0, 1)
I run the following code in R:
cvfit.retro = cv.glmnet(retro.x, y=as.factor(retro.y), family='binomial')
where retro.x is the matrix of 247 predictors retro.y is the vector of the binomial variable
The idea behind running the code was to reduce the number of variables to a more manageable number for further analysis for their predictive power in relation to the binomial dependent variable.
However, the cvfit.retro output at a relatively high lambda of exp(-6),
shows highly correlated variables being INCLUDED in the model. For example, there are two variables called v708 and v709 that have non-zero coefficients in the CV.GLMNET output and these two variables have a correlation of 0.9995 using the cor() function. These variables are obviously powerful predictors in the model but one of them should have been zeroed out in the LASSO in CV.GLMNET given that alpha defaults to 1 (LASSO) in cv.glmnet
Am I missing something here? Can someone explain how I can fix this issue? Obviously, I could just eliminate one of the correlated variables manually but that kind of partially defeats the purpose of LASSO. Any advice is appreciated. Thank you!