# Using LASSO only for feature selection

In my machine learning class, we have learned about how LASSO regression is very good at performing feature selection, since it makes use of $l_1$ regularization.

My question: do people normally use the LASSO model just for doing feature selection (and then proceed to dump those features into a different machine learning model), or do they typically use LASSO to perform both the feature selection and the actual regression?

For example, suppose that you want to do ridge regression, but you believe that many of your features are not very good. Would it be wise to run LASSO, take only the features that are not near-zeroed out by the algorithm, and then use only those in dumping your data into a ridge regression model? This way, you get the benefit of $l_1$ regularization for performing feature selection, but also the benefit of $l_2$ regularization for reducing overfitting. (I know that this basically amounts to Elastic Net Regression, but it seems like you don't need to have both the $l_1$ and $l_2$ terms in the final regression objective function.)

Aside from regression, is this a wise strategy when performing classification tasks (using SVMs, neural networks, random forests, etc.)?

• Yes, Using lasso for feature selection for other models is a good idea. Alternatively tree based feature selection could also be fed to other models – karthikbharadwaj May 9 '16 at 23:31
• The lasso only performs features selection in linear models -- it doesn't test for higher-order interactions or nonlinearity in the predictors. For an example of how that might be important: stats.stackexchange.com/questions/164048/… Your mileage may vary. – Sycorax May 10 '16 at 1:02