# What to do when lasso does not remove correlated variables?

The very essence of lasso is that it is supposed to select only one of two correlated variables.

However, when I include two highly correlated predictors (they are correlated with each other at level ~0.95), both of them are being selected with similar absolute coefficient value (on standardized predictor), but with different signs. This means their effect on prediction almost cancels out, but the coefficients from model on standardized input are highest of all variables.

Example:

          x         coefs
(Intercept)        91.6958266
Population_2013   -49.2656083
Population_2014    46.8513210


where Variable1 and Variable2 are highly correlated. Other correlated and uncorrelated variables are also included in model. I run models on anything between 20 and 20000 variables and effect is similar for these correlated variables.

Is there any solution? Alternatively - how in any other way can I determine which variables affect significantly my prediction?

• Use ridge regression or elastic net when you have highly correlated variables. – Richard Hardy May 20 '16 at 16:17
• What is the objective of your analysis? And what do you mean "run models on anything between 20 and 20,000 variables"? I don't agree that "the very essence of lasso is that it is supposed to select only one of two correlated variables". Yes, the lasso is relatively unstable with correlated variables (small changes in $Y$ can change the solution), but that is only a side effect of the $\ell_1$ penalty on the sum of squares, not a desideratum. – Andrew M May 20 '16 at 16:40
• From Richard Hastie's glmnet package vignette: "It is known that the ridge penalty shrinks the coefficients of correlated predictors towards each other while the lasso tends to pick one of them and discard the others." - this is what I mean by essence of lasso w.r.t. correlated variables. I don't understand how ridge regression can help here (actually - it does not change anything). I run models on different business financial indicators with initially large number of automatically generated hypothesis to test, I delete these I cannot attribute meaningful causality to my dependent variable. – user2530062 May 21 '16 at 20:36
• Actually someone else suggested to me it might be lack of penalization - lambda chosen by algorithm to minimize CV error is too low and correlated variables are being allowed into the model. I will investigate it on Monday, I will appreciate other suggestions though. – user2530062 May 21 '16 at 20:44