Variable selection in large datasets

Question

I'm looking for an overview of some methods of variable selection.

I use datasets with around 6000 variables (the level of missing values is satisfying i.e. there are no variables with 100% missing values). There is no problem of insufficient degrees of freedom in my datasets.

In order to use some data mining techniques (such as regression, gradient boosting, etc) I use some methods of removing "weak" variables.

I am aware of stepwise methods for regression, but I'm looking for use with broader range of methods.

Does anyone know of any scientific papers or other articles about this subject?

Answer 1

Here's a nice review of the topic. It's addressed to the Bioinformatics field, but the concepts generalize to other applications. http://bioinformatics.oxfordjournals.org/content/23/19/2507.abstract

Answer 2

Yes, there are several books. Here is my list:

• Model Selection and Inference: A Practical Information-Theoretic Approach by Burnham and Anderson Springer 1998.
• Model Selection by Linhart and Zucchini Wiley 1986.
• Regression and Time Series Model Selection, McQuarrie an Tsai World Scientific 1998.
• Model Selection, Institute of Mathematical Statistics collection of papers edited by P. Lahiri Vol 38 in their Lecture Notes-Monograph Series.

Lacey Gunter and I wrote a monograph on it for Springer that will appear later this year in their new SpringerBrief series. Our book briefly summarizes the material in the aforementioned books but mainly covers the new qualitative interaction approach to variable selection that I describe in a question I posted here yesterday.