How to combine stability selection and model selection with LASSO? I was reading about stability selection applied to LASSO. My understanding is that stability selection (Meinshausen & Buhlman, 2010) helps in finding stable variables, with error control provided by an upper bound, but it doesn't provide an optimized regularisation parameter, so I'd say that it doesn't really do model selection.
I see how using only the subset of stable variables identified in this way to train further models might be wrong. So my question is how feature selection performed with stability can be integrated in a predictive framework?
Imagine I have a classification problem with binary outcome. I am interested in assessing the prediction performance of LASSO and identify stable predictors and their sign (association with the outcome).
The only way I see is to perform a LASSO using, for instance,  a nested k-fold cross-validation and evaluate model performance.
Then perform stability selection as a separate analysis to identify significant predictors.
I expect that the subset of variables selected in each loop of the nested cv will be, at least, slightly different from the stable variables identified by stability selection.
My question is, is it ok to evaluate model performance with a metod and predictors importance with another method? Is there a way to extract coefficient signs from stability selection procedure (I'm using stabsel in R).
 A: I am thinking about stability selection more like a statistical procedure to give you a set of significant variables in a high dimensional setting, where traditional linear regression would fail. That it uses LASSO in the paper, which is usually used for machine learning purposes is kind of a red herring.
Now, as you correctly noticed, it cannot be easily used for model selection, but you can use it for feature selection, where the selected features by stability selection will be likely better than features selected by LASSO alone. In this case, you will first select variables using stabsel, and then fit a model that does not have to be LASSO, since you don't want to do another feature selection on these variables alone. You don't need to only take the selected variables, since the stabsel selection is quite conservative. Just be sure you do it inside your CV loop to avoid double-dipping.
Another possibility is to run and validate your LASSO the same way as you do. But then as a supplementary analysis also run stability selection to tell you which variables are significant. Since prediction and identifying statistically significant variables are separate goals, you can have separate analyses for that.
ps: stability selection works for any feature selection method, not just LASSO, lasso is just very convenient since it can produce the whole regularization path in one go
