Which approach based on the LASSO yields more biologically relevant results for gene data-sets?

Question

I have a data-set with a continuous outcome variable and some confounding variables (like age, gender, ...) and many gene expressions (more than samples). The goal is to find relevant genes in association with the outcome.

Now the first idea was to use the LASSO (Tibshirani 1996). Some questions arose regarding the whole procedure.

1. Does one include the confounding variables in the variable selection stage and keep them in the model without regularization? I have seen that including such fixed variables changes the selected genes.
2. In order to only select stable genes I used the stability selection procedure (Meinshausen and Bühlmann 2010). Does one need confidence intervals in this procedure or only in the basic LASSO?
3. Would it also make sense to use some LASSO generalization (like group LASSO or newer ideas) to look for relevant networks/groups of genes instead of single genes? Or could one look for associated genes with the selected genes from LASSO, in order for the results to be more interpretable (e.g. clustering by correlation, nodewise regression, group LASSO with groups based on correlation...)? Can this be done on the same data-set or are new measurements needed?
4. Can the residuals of the LASSO model be analysed? Or does one construct an ordinary regression with the selected variables and look at that model? Or what is the procedure here?
5. Which other approach would you suggest?

Answer 1

1. Since you're only interested in the influence of the genes, you should "force keep" the non-gene variables in the model (for example in R with glmnet package and option penalty.factor equal to zero for the corresponding variables). Or you could first do a model with the confounding variables only and identify the ones that have a significant influence on the outcome and then force-keep these along with the genes in the second full model. Another approach would be to stratify your data such that you're doing different lasso models for subgroups (given that you have enough observations), e.g. which genes are chosen for females/age 18-49 compared to males/age 18-49 etc.
2. Confidence intervals aren't needed, as pointed out by rep_ho in the comment to the question.
3. You CAN use group lasso or sparse-group lasso if you have pathway/subnetwork information on the genes available. Be aware that vanilla lasso has an undesirable property: if some features are correlated, it tends to choose one of them and disregards the other ones in the remainder. This translates to group lasso but on the group level (out of some correlated groups it chooses only one group) and to sparse-group lasso as well (but here for correlations on between- and within-group levels). It depends on the original scientific question (respectively, what your outcome is) if you're more interested in specific genes or specific pathways. Additionally, stability selection is already a good idea to deal with correlated variables. Furthermore, you might want to take a look into elastic net regression, which is basically a mixture of lasso and ridge regression and has been shown to handle correlated variables better than lasso.
4. I'm not sure what you're referring to. I hope you're doing cross-validation, then you're actually assessing the residuals to find the best $\lambda$ value and as such the corresponding regression coefficients that best describe your outcome and avoid overfitting. You shouldn't do an additonal model with the chosen features, instead analyse the model chosen by cross-validation.
5. Lasso is fine. Elastic net might be worth a look. Some Support Vector Machine models (e.g. SVEN) are shown to be similar to lasso methods, and there's Bayesian alternatives to the different lasso methods as well which might be more precise than lasso ("spike-and-slab", for an overview and slow implementations based on Gibbs sampling see here, and more efficient implementations: simple spike-and-slab, grouped spike-and-slab, and sparse-group spike-and-slab). (Disclaimer: the last reference is a paper by me.)