How to make a selection of a specific number of features using Lasso?

Question

I have a project to do in which I need to select the 5 most valuable features using Lasso. I wonder what is the right approach to select a specific number of features using Lasso. Searching the Internet, I've seen people make feature selections in such a way that they look for the best regularization factor for Lasso using, for example, GridSearch, and then for that factor they select those features that are not zeroed out. In this way, however, it is not possible to select a specific number of features, only as many as will not be zeroed out. What should I do to select only 5 features?

I am considering two ways to do this:

• Selecting the best regularization factor using GridSearch and for the best model, selecting the 5 features with the largest coefficients (in terms of absolute value).
• Finding a regularization factor that zeros out all features except 5 and selecting those 5 features.

Could someone tell me which approach makes more sense?

Answer 1

The lasso wants you to use the model that has optimum cross-validated deviance. Use however many variables that entails. But note that lasso is notoriously bad (just as most other variable selection methods are bad) in choosing the right variables. See the high-dimensional data analysis chapter in BBR for examples, as well as this.

Consider data reduction methods (unsupervised learning) instead of variable selection. RMS has some material on this.