Regression in $p\gg N$ setting (predicting drug efficiency from gene expression with 30k predictors and ~30 samples) I have a dataset of 29 cell lines and the IC50 values of a test drug. I want to find a relation between the gene expression profiles of each cell line (nearly 31000 genes) and the IC50 values.
My problem is the huge number of independent variables (the genes) and the low number of samples (cell lines). I'm trying to perform a linear regression using Lasso to reduce the number of genes, dividing the samples in a train set of 14 cell lines and a test set of 15 cell lines. The division is performed by randomly sampling among the 29 samples. The problem is that Lasso is not stable and every time I train the model I get different results.
So I tried to reduce the dimensionality using PCA, but as far as I have read, PCA doesn't work well when the number of covariates is greater then the number of samples. Is this true?
Can you suggest me some kind of regression which is robust when the number of samples is low?
 A: Most of the 31000 genes are unlikely to differ much in expression among the cell lines (at least when appropriately normalized), so they add no information to the problem.
For a practical biological problem like this, it may help to concentrate on genes whose expression levels are relatively high on an absolute basis. That way it will be easier to validate your results on these 29 lines and then apply and test your predictions on cell lines beyond those you are now examining, for example with standard PCR instead of the expensive microarray or RNAseq methods used to examine 31000 genes at once.
Start by (a) limiting your analysis to highly expressed genes whose normalized expression levels have the greatest variance among cell lines (typically on a log scale in gene expression work) and closest relation to IC50 values, so that your intractable $p \gg n$ problem becomes a less difficult $p > n$ problem. Then (b) combine information from different genes whose expression levels co-vary among cell lines. 
The "supervised principal components" method described in section 18.6 of The Elements of Statistical Learning, second edition, provides a documented way to accomplish this. Genes are rank-ordered with respect to univariate relations to IC50 values (accomplishing goal a, if you limit to highly expressed genes) and PCA is performed on a subset of genes with the highest relations to IC50s (accomplishing goal b). The number of genes included in the PCA and the number of principal components retained are chosen by cross-validation. 
A: Can I suggest you the paper robustness of lasso solutions under cross-validation variability and Stability selection (Meinshausen and Bruhlman, 2009)?
They propose stable version of the lasso estimator.
A: I believe the reason you will be getting varying answers is because you have $p>>n$, i.e more variables than samples. In this situation the LASSO can only selected $n$ variable and I assume will have problems with convergence. While having no experience with dealing with this, the Elastic Net supposedly overcomes some of these issues
