Feature selection & Cross Validation

this is a popular topic here but I have been reading through the different pages and could not find anything related with what I am wondering now.

So, I have a data set with X features and I would like to obtain a subset of them that are related with the output variable (aka feature selection). Let's say I want to use Lasso regression model with L1 regularization.

My question is: How should I apply CV in here?

Should I count the number of times a specific variable is discarded in each of the folds? At the very end I would know which of them were more discarded... But I do not trust on this approach since I am discarding the variance of my results. I guess I should apply some hypothesis testing to have some certainty on my results, but what should I apply?

Thank you all!

• Fix some value of $\lambda$. On some of the data, estimate the lasso coefficients. Then, on the rest of the data, compute the prediction error. Do this a bunch of times over all of the folds. Do this repeatedly for all of the values of $\lambda$ that you're interested in. Whichever $\lambda$ has the lowest average (over all folds) prediction error wins. – user795305 Aug 29 '17 at 14:33