This is a rather general question. If the question is vague and hard to answer in a few lines, I'd be happy if someone just point me to some readings. Thanks in Advance.
I am working on a multi-class classification problem with a large datasets (>3000 variables) and I am looking to reduce the dimensionality. My first step is to using "filter" methods to evaluate the relevance of a single feature and the target variable to eliminate some very low score features.
I am planning on just using Mutual information as my relevance score, but as I read about it I realize there are more methods such as Fisher score, Gini index or simply correlation coefficients.
Q: Is there a better relevance measure for the given classifier I'd use? Eg. If I decide to use SVM, or decision tree as my classifier, would one of the relevance measure be more optimal than the others?
EDIT: I understand there are better methods (eg, some wrapper methods) in practice, but I am more interested in the theoretical analysis of how these filter methods mesh with the classifiers.