I'm looking for an overview of some methods of variable selection.

I use datasets with around 6000 variables (the level of missing values is satisfying i.e. there are no variables with 100% missing values). There is no problem of insufficient degrees of freedom in my datasets.

In order to use some data mining techniques (such as regression, gradient boosting, etc) I use some methods of removing "weak" variables.

I am aware of stepwise methods for regression, but I'm looking for use with broader range of methods.

Does anyone know of any scientific papers or other articles about this subject?

  • $\begingroup$ In addition to feature-selection that I added, you will probably find useful reference by looking at model-selection-related threads. $\endgroup$
    – chl
    May 5, 2012 at 20:26

2 Answers 2


Here's a nice review of the topic. It's addressed to the Bioinformatics field, but the concepts generalize to other applications. http://bioinformatics.oxfordjournals.org/content/23/19/2507.abstract


Yes, there are several books. Here is my list:

  • Model Selection and Inference: A Practical Information-Theoretic Approach by Burnham and Anderson Springer 1998.
  • Model Selection by Linhart and Zucchini Wiley 1986.
  • Regression and Time Series Model Selection, McQuarrie an Tsai World Scientific 1998.
  • Model Selection, Institute of Mathematical Statistics collection of papers edited by P. Lahiri Vol 38 in their Lecture Notes-Monograph Series.

Lacey Gunter and I wrote a monograph on it for Springer that will appear later this year in their new SpringerBrief series. Our book briefly summarizes the material in the aforementioned books but mainly covers the new qualitative interaction approach to variable selection that I describe in a question I posted here yesterday.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.