I am developing a feature selection method based on a LASSO regression model.

The coefficient are as follow:


I want to know how to select the best features in my model. Should it based on highest values or the highest of absolute (abs) values?

  • $\begingroup$ If I'm understanding your post then the coefficients are resulting from a LASSO procedure. If that is the case then the feature selection is already performed (c.f one of the coefficients has been shrinked to 0 meaning it's been pruned off the model) $\endgroup$ – Riff Jun 29 '17 at 8:02
  • $\begingroup$ Yes, exactly, some of them have been shrinked to 0 but I need to select the the first 5 important features. I guess , I maybe sort them based on the abs of coefficients. $\endgroup$ – Luckylukee Jun 30 '17 at 8:26
  • $\begingroup$ You could then tweak your penalty factor until no more than 5 variables are different than 0. The problem with what you're tempted to do is that the coefficients you are observing now are totally dependent of your lambda choice so it is not just a matter of selecting the higher values because if you flush some variables out you will have a totally different set of coefficients $\endgroup$ – Riff Jun 30 '17 at 9:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.