For high-dimensional molecular genetic data, is there a difference in available feature selection techniques between classification problems and regression problems? Or can all feature selection techniques be applied to either classification and regression modeling indiscriminately?


1 Answer 1


There is a huge difference. Classification has the efficiency of the sign test at best ($\frac{2}{\pi}$) whereas prediction can use all the information in the data and will work better on new samples. When using classification, the entire classification scheme may have to be re-done from scratch if you alter the outcome prevalence through oversampling.

Classification uses an improper accuracy scoring rule which is easily fooled into selecting the wrong features.

For more detail see Biostatistics for Biomedical Research Sections 18.3.5 and 18.4 and listen to the audio.

  • $\begingroup$ Thans for the answer Frank. In the second sentence, I think you meant to say regression instead of prediction, right? $\endgroup$
    – Ansjovis86
    Mar 31, 2016 at 14:21
  • $\begingroup$ No I meant prediction, but that is often done with regression. $\endgroup$ Mar 31, 2016 at 15:29
  • $\begingroup$ Hi Frank, could you recommend some feature selection techniques for regression problems? $\endgroup$
    – Ansjovis86
    Apr 11, 2016 at 15:30
  • $\begingroup$ There are not many that I would recommend whether using regression or some other method, because the data are usually incapable of telling you which features are "important". If you have to use feature selection, i.e., if you are less interested in prediction per se, I'd recommend methods that combine appropriate shrinkage with selection, e.g., the elastic net. $\endgroup$ Apr 11, 2016 at 15:55
  • $\begingroup$ It's microarray data so I have to perform some kind of feature selection, because otherwise the Machine Learning part for predicting will take a long time to compute with half a million features. $\endgroup$
    – Ansjovis86
    Apr 11, 2016 at 15:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.