In order to test a potential classification set, usually some data is kept as a holdout set, and not used for inner-cross-validation or model training.
However, what happens if too many features exist, and it would be impossible to do iterative cross-validated feature selection on all of them.
Is it acceptable (and scientific) to first reduce the overall number of features using Filter Based Feature Selection (e.g. Pearson Correlation)?
Is it also acceptable to further reduce the number of features / dimensions using Fisher Linear Discriminant Analysis (LDA)?
As for acceptable and scientific, I mean, if cross-validated feature selection is still done on the remaining features, then would information still be leaked into the test set causing a classification performance bias?