# Regression in $p\gg N$ setting (predicting drug efficiency from gene expression with 30k predictors and ~30 samples)

I have a dataset of 29 cell lines and the IC50 values of a test drug. I want to find a relation between the gene expression profiles of each cell line (nearly 31000 genes) and the IC50 values.

My problem is the huge number of independent variables (the genes) and the low number of samples (cell lines). I'm trying to perform a linear regression using Lasso to reduce the number of genes, dividing the samples in a train set of 14 cell lines and a test set of 15 cell lines. The division is performed by randomly sampling among the 29 samples. The problem is that Lasso is not stable and every time I train the model I get different results.

So I tried to reduce the dimensionality using PCA, but as far as I have read, PCA doesn't work well when the number of covariates is greater then the number of samples. Is this true?

Can you suggest me some kind of regression which is robust when the number of samples is low?

• PCA is probably not the best way to approach your problem, but just for the record: PCA does work fine in $p>n$ situation, or at least not less "fine" than anything else. – amoeba says Reinstate Monica Dec 8 '14 at 22:50
• – amoeba says Reinstate Monica Dec 30 '14 at 13:12

Most of the 31000 genes are unlikely to differ much in expression among the cell lines (at least when appropriately normalized), so they add no information to the problem.

For a practical biological problem like this, it may help to concentrate on genes whose expression levels are relatively high on an absolute basis. That way it will be easier to validate your results on these 29 lines and then apply and test your predictions on cell lines beyond those you are now examining, for example with standard PCR instead of the expensive microarray or RNAseq methods used to examine 31000 genes at once.

Start by (a) limiting your analysis to highly expressed genes whose normalized expression levels have the greatest variance among cell lines (typically on a log scale in gene expression work) and closest relation to IC50 values, so that your intractable $p \gg n$ problem becomes a less difficult $p > n$ problem. Then (b) combine information from different genes whose expression levels co-vary among cell lines.

The "supervised principal components" method described in section 18.6 of The Elements of Statistical Learning, second edition, provides a documented way to accomplish this. Genes are rank-ordered with respect to univariate relations to IC50 values (accomplishing goal a, if you limit to highly expressed genes) and PCA is performed on a subset of genes with the highest relations to IC50s (accomplishing goal b). The number of genes included in the PCA and the number of principal components retained are chosen by cross-validation.

• +1 Doesn't what you are describing (find genes with most variation; find groups of genes that are highly correlated) sound a lot like PCA? Would you still advice against PCA but in favour of such "manual" feature selection? Why? – amoeba says Reinstate Monica Dec 9 '14 at 15:46
• Each of the principal components found by PCA could in principle contain contributions from all 31000 genes, which could be a computational nightmare and would be difficult (at best) to interpret functionally. The biological questions underlying this type of analysis are best analyzed by focusing on a few genes or networks of genes. – EdM Dec 9 '14 at 15:56
• Interesting. So maybe sparse PCA (with very strong sparsity enforced) could be an interesting approach. – amoeba says Reinstate Monica Dec 9 '14 at 15:58
• The approach that The Elements of Statistical Learning calls "supervised principal components" is also possible here, and close to what I suggested: find the subset of genes whose expression levels have highest univariate correlations to IC50 levels, and restrict PCA to those genes. The number of genes included in the PCA and the number of principal components retained are chosen by cross-validation. – EdM Dec 9 '14 at 16:07
• Yes, section 18.6 of The Elements is indeed very relevant, thanks for the pointer. Maybe you want to update your answer to add what you said in the comments. – amoeba says Reinstate Monica Dec 9 '14 at 16:32

I believe the reason you will be getting varying answers is because you have $p>>n$, i.e more variables than samples. In this situation the LASSO can only selected $n$ variable and I assume will have problems with convergence. While having no experience with dealing with this, the Elastic Net supposedly overcomes some of these issues

Can I suggest you the paper robustness of lasso solutions under cross-validation variability and Stability selection (Meinshausen and Bruhlman, 2009)?

They propose stable version of the lasso estimator.

• @Donbea: can you give a bibliographically correct reference? (scientific articles often have authors, date of publications and sometimes even journals) – user603 May 29 '14 at 13:48
• stats.ox.ac.uk/~meinshau/stability.pdf – Donbeo May 29 '14 at 14:18
• Google Scholar search for "robustness of lasso solutions" returns zero hits. What paper were you referring to? – amoeba says Reinstate Monica Dec 8 '14 at 22:49
• I can not find the paper anymore. Maybe I have reported a wrong name. Anyway I will say that Stability Selection should definitive be the important reference – Donbeo Dec 8 '14 at 22:53