I have used lasso logistic regression on some data and I have some non zero coefficients for some of the features.
I want to know based upon the coefficient values how do I rank the features?
$\begingroup$
$\endgroup$
3
-
1$\begingroup$ To my knowledge you can interpret the coefficients generated through a lasso procedure the same as you would those generated through a standard logistic regression. $\endgroup$– BGreeneCommented Feb 17, 2014 at 14:28
-
$\begingroup$ @BGreene. But how do I rank the coefficients? $\endgroup$– user34790Commented Feb 17, 2014 at 14:36
-
$\begingroup$ Well the size of the coefficient is indicator of effect size - the coefficient is equivalent to the log of the odds ratio. So you can rank by coefficient size. $\endgroup$– BGreeneCommented Feb 17, 2014 at 15:18
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Why do you want to rank the features ? In order to have some variable selection at the end ?
If so, if your goal is to rank features according to an "importance measure" using the LASSO you may want to look at stability selection : http://stat.ethz.ch/~nicolai/stability.pdf
In this paper, the authors propose a methods to rank the features :
For each bootstrap sample :
- run the LARS algorithm for the LASSO (this is a homotopic algorithm, ie. you know the entire path of regularisation)
- keep the lambda value (regularization parameter) for which each variable become non zero in the regularization path.
Calculate the expected value of this non zero lambda for each value. You then have a way of ranking variables, from high lambda to small lambda.
You should be aware that very correlated features will probably have a very similar rank at the end.
