I am using LASSO (glmnet) to do feature selection.
However, how can I check which features are selected?
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.Sign up to join this community
coef function on the glmnet model.
You will need to choose a lambda value, as different lambdas will give you different feature sets. Typically this is done through cross-validation.
/edit: For example, using
library(glmnet) x <- model.matrix(Sepal.Length~., iris)[,-1] y <- iris$Sepal.Length mod <- cv.glmnet(as.matrix(x), y, alpha=1)
To see the coefficients with the minimum cross-validation error:
as.matrix(coef(mod, mod$lambda.min)) 1 (Intercept) 2.1670759 Sepal.Width 0.5032347 Petal.Length 0.8137398 Petal.Width -0.3127065 Speciesversicolor -0.6763395 Speciesvirginica -0.9595409
To see the coefficients with the "largest value of lambda such that error is within 1 standard error of the minimum:"
as.matrix(coef(mod, mod$lambda.1se)) 1 (Intercept) 2.14705035 Sepal.Width 0.59950383 Petal.Length 0.57550203 Petal.Width -0.23632776 Speciesversicolor 0.00000000 Speciesvirginica -0.04770282
You can also select any other value of lambda that you want. Coefficients that are 0 have been dropped out of the model. e.g.:
CF <- as.matrix(coef(mod, mod$lambda.1se)) CF[CF!=0,] (Intercept) Sepal.Width Petal.Length Petal.Width Speciesvirginica 2.14705035 0.59950383 0.57550203 -0.23632776 -0.04770282
If we uses the 1se lambda, the Speciesversicolor dummy variable gets dropped from the model.