I am working on a biomedical/healthcare data science problem.

I have a dataset of 600 samples, ~6000 variables and class label as "positive" or "negative". I want to perform feature selection on this high-dimensional data. I want to know what are the best strategies to find which feature is most contributing here. Typically, I design a binary classifier (SVM, NB, RF or ANN) and then use information gain or RandomForest to assess the feature importance. Here, my variable space is ten-times more than my instances - hence am not comfortable with this approach.

Is there any better strategy to reduce the feature space in a data-driven way?

  • 1
    $\begingroup$ What about trying dimensionality reduction? $\endgroup$ – Aleksandr Blekh Apr 11 '15 at 1:59
  • 1
    $\begingroup$ Not data-driven? So, on theoretical basis maybe? Don't think that is a good idea. I would try something like Supervised Principal Components. $\endgroup$ – MaHo Apr 11 '15 at 6:21
  • 1
    $\begingroup$ Why not try lasso regression. It has been developed specifically for those kind of problems, fewer data than dimensions) (eg using glmnet package in R).. $\endgroup$ – seanv507 Apr 11 '15 at 8:03
  • $\begingroup$ Thanks @AleksandrBlekh for suggestion on dimensionality reduction. I was using one of such approach, but wanted to see what are best way test all features without overfitting the model. $\endgroup$ – Khader Shameer Apr 13 '15 at 15:02
  • 1
    $\begingroup$ I see. However, unless it is customary in your subject domain, I'm not sure that it is feasible to have and analyze so high-dimensional models. $\endgroup$ – Aleksandr Blekh Apr 13 '15 at 19:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.