Which predictors to include in multiple regression where predictors are more than number of samples? I have 3000 predictors (biological measurements) and 200 samples (weight), and would like to build a linear multiple regression model. How do I select which predictors to include in the regression model? 
Someone suggested using R package 'rpart'.
Please help.
 A: carmen, I have three suggestions for you:


*

*Use the R package glmnet to estimate the coefficients. This package uses a penalize likelihood that shrinks many coefficients to zero. It's helpful not only for having more predictors than observations, but also for dealing with multicollinearity among your predictors. Note that you might have 3000 predictors, but you could have many more features than that if you include polynomial terms (I recommend not using any more than three) and interactions. glmnet has no problem with that. You will, however, need to use k-fold cross-validation to tune the likelihood penalization parameter, and the parameter that determines the weight of LASSO versus ridge penalization.

*Use the R package randomForest if you wish to avoid the choice of what interaction effects and polynomials to include altogether. Random forest creates many decision trees based on boostrapped samples of your data. You will need to tune the mtry parameter using cross-validation. This parameter guides the number of features that are chosen at random to create each tree in the forest. Also, I recommend you grow a large forest of at least 200 trees.

*Consider variable reduction using something like Principle Components Analysis. This is especially useful if you can identify certain subsets of your predictors that are highly correlated (a bit of a challenge since that involves computing a 3000x3000 correlation, but that's nothing R can't handle if you've got enough RAM), and maybe also related to one another (maybe on the same metabolic pathway or something). For each case of principle components analysis, include principle components up until the point where 85-95% of the total variance is "explained".
