I am performing elastic net for variable selection on a dataset of 95 records and 41 variables. The response is a continuous numerical.
I choose the alpha and lambda parameters through 10 fold cross validation with
glmnet package of R.
I get alpha = 0.75 and lambda = 0.0125685859783604 (I decided to pick lambda.1se, instead of lambda.min).
42 x 1 sparse Matrix of class "dgCMatrix" s0 (Intercept) -2.296544e+02 V1 3.940156e-01 V2 . V3 3.082747e-01 V4 -1.683031e-01 V5 -1.451309e+00 V6 1.448475e+00 V7 1.495223e+00 V8 2.578494e+00 V9 7.921473e+01 V10 -8.396766e+01 V11 -5.018264e+02 V12 1.198209e+00 V13 8.529590e+00 V14 -1.196211e+01 V15 -3.316756e+00 V16 -5.806878e+00 V17 -2.276801e+02 V18 . V19 4.525065e+05 V20 1.307480e-02 V21 1.632396e+06 V22 . V23 . V24 -1.070972e-03 V25 -3.382517e-01 V26 2.897391e-02 V27 . V28 2.289668e+01 V29 -9.895771e+01 V30 -8.814758e-02 V31 . V32 . V33 3.369844e+06 V34 1.560076e-03 V35 2.800003e+05 V36 . V37 -6.399852e+01 V38 . V39 -6.388111e-01 V40 2.164014e-02 V41 .
The coefficients assigned to V19, V21 and V33 are very high, but checking on the correlogram these three variables are also highly correlated.
My question is: does elastic net deal with collinearity? With alpha near 0 (ridge regression) I should expect that all variables are retained, but since alpha is nearer to 1 (lasso regression), why highly correlated variables were mantained in the model? Is an expected behaviour - highly correlated variables get similar coefficients - or not?