2
$\begingroup$

I am performing elastic net for variable selection on a dataset of 95 records and 41 variables. The response is a continuous numerical. I choose the alpha and lambda parameters through 10 fold cross validation with glmnet package of R.

I get alpha = 0.75 and lambda = 0.0125685859783604 (I decided to pick lambda.1se, instead of lambda.min).

42 x 1 sparse Matrix of class "dgCMatrix"
                       s0
(Intercept) -2.296544e+02
V1           3.940156e-01
V2           .           
V3           3.082747e-01
V4          -1.683031e-01
V5          -1.451309e+00
V6           1.448475e+00
V7           1.495223e+00
V8           2.578494e+00
V9           7.921473e+01
V10         -8.396766e+01
V11         -5.018264e+02
V12          1.198209e+00
V13          8.529590e+00
V14         -1.196211e+01
V15         -3.316756e+00
V16         -5.806878e+00
V17         -2.276801e+02
V18          .           
V19          4.525065e+05
V20          1.307480e-02
V21          1.632396e+06
V22          .           
V23          .           
V24         -1.070972e-03
V25         -3.382517e-01
V26          2.897391e-02
V27          .           
V28          2.289668e+01
V29         -9.895771e+01
V30         -8.814758e-02
V31          .           
V32          .           
V33          3.369844e+06
V34          1.560076e-03
V35          2.800003e+05
V36          .           
V37         -6.399852e+01
V38          .           
V39         -6.388111e-01
V40          2.164014e-02
V41          .         

The coefficients assigned to V19, V21 and V33 are very high, but checking on the correlogram these three variables are also highly correlated.

enter image description here

My question is: does elastic net deal with collinearity? With alpha near 0 (ridge regression) I should expect that all variables are retained, but since alpha is nearer to 1 (lasso regression), why highly correlated variables were mantained in the model? Is an expected behaviour - highly correlated variables get similar coefficients - or not?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.