I'm doing classification with two classes. Before I apply a classifier, I'm doing some preprocessing steps like removing near-zero variance features or highly correlated features (for those classifiers which are sensitive to it).

Now I will also add a feature selection step (ReliefF and genetic algorithms). Do I still have to do the above preprocessing steps before or after the feature selection or is this already incorporated in the feature selection? I think the feature selection process should already eliminate correlated and near-zero variance features but I'm not completely sure. Of course standardization and missing value imputation I have to do before the feature selection.

  • $\begingroup$ Are you sure your feature selection algorithm won't crash with these predictors? If not, then you might as well since there's no harm in doing it before running the algorithm. $\endgroup$ – dsaxton Apr 5 '16 at 13:38
  • $\begingroup$ @dsaxton No, I'm not sure if it crashes or not. But why should it crash? For example, genetic algorithms use the underlying classifier which e.g. works bad for correlated predictors, so these features get removed. $\endgroup$ – BlackHawk Apr 5 '16 at 15:59

Looks like you already gave the answer. In theory, any automated variable/feature selection method should replace manual preprocessing work. In your case, I can't imagine how something as powerful as multi-class Relief with a Genetic Algorithm could fail to detect high correlations. I like having machines do the thinking so I would personally skip manual work but nothing stops you from trying both ways and comparing the results.

  • $\begingroup$ I'm thinking about doing ReliefF or Genetic Algorithm but not both in sequence. Would you do them in sequence? By the way, I only have 20 features... $\endgroup$ – BlackHawk Apr 5 '16 at 16:00
  • $\begingroup$ Not in sequence, I thought you were going to combine them. I would try them out separately. $\endgroup$ – Digio Apr 6 '16 at 8:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.