2
$\begingroup$

I am using LASSO (glmnet) to do feature selection.

However, how can I check which features are selected?

$\endgroup$
4
$\begingroup$

Use the coef function on the glmnet model.

You will need to choose a lambda value, as different lambdas will give you different feature sets. Typically this is done through cross-validation.

/edit: For example, using cv.glment:

library(glmnet)
x <- model.matrix(Sepal.Length~., iris)[,-1]
y <- iris$Sepal.Length
mod <- cv.glmnet(as.matrix(x), y, alpha=1)

To see the coefficients with the minimum cross-validation error:

as.matrix(coef(mod, mod$lambda.min))
                           1
(Intercept)        2.1670759
Sepal.Width        0.5032347
Petal.Length       0.8137398
Petal.Width       -0.3127065
Speciesversicolor -0.6763395
Speciesvirginica  -0.9595409

To see the coefficients with the "largest value of lambda such that error is within 1 standard error of the minimum:"

as.matrix(coef(mod, mod$lambda.1se))
                            1
(Intercept)        2.14705035
Sepal.Width        0.59950383
Petal.Length       0.57550203
Petal.Width       -0.23632776
Speciesversicolor  0.00000000
Speciesvirginica  -0.04770282

You can also select any other value of lambda that you want. Coefficients that are 0 have been dropped out of the model. e.g.:

CF <- as.matrix(coef(mod, mod$lambda.1se))
CF[CF!=0,]
 (Intercept)      Sepal.Width     Petal.Length      Petal.Width Speciesvirginica 
  2.14705035       0.59950383       0.57550203      -0.23632776      -0.04770282 

If we uses the 1se lambda, the Speciesversicolor dummy variable gets dropped from the model.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.