I'm working with a Single-nucleotide polymorphism (SNP) dataset with over 2.2 million features and roughly 2000 samples.
I wish to do feature selection on this dataset to reduce it down to approximately ~20,000 features so that I can then use them to train different classifiers for a binary classification problem. The 2 classes are in the ratio of 3:1
What are the best feature selection methods in this scenario?
You might consider doing LD (Linkage Disequilibrium) SNP pruning to get your feature set down to a more manageable level. The idea is that lots of neighboring SNPs are very highly correlated. So often only SNPs that are in LD are considered.
This probably won't get it down to 20,000 but its a good start that you will see in a lot of the GWAS literature.
You might consider a penalized regression approach (glmnet in R) which will do some automatic variable selection as well as multinomial regression.
For SNP data I highly recommended Sean Purcell's PLINK program.
As a side note, I would also do PCA on the SNPs and incorporate the first few leading PCs to control for ethnicity (called population stratification in the genetics literature).
Start with some simple pairwise measure/test and see how that does.
Random forests are commonly used for genomic data and can be used for feature selection (importance scores). Variations on this methods life artificial contrasts (ACE), using out of bag cases to valuate splitter importance and roughly balanced bagging or weighted rf help deal with some of the oddities of data sets you describe.
It is also common to calculate gene or pathway burden scores/summaries which can be as simple as summing the number of non reference SNPs in each gene or calculating a distance from the mean.
Summarizing by pathway is a really interesting and promising field and there are a number of papers out there on "subnetwork markers" or pathway burden.
Finally methods like Eigenstrat may be interesting because they correct for confounding population stratification by doing eigen decomposition dimensionality reduction and identifying dimensions that have to do with admixture etc. I'm not sure if you can just take the top n non confounding dimensions after correction for further analysis but it would be worth a try.
