I am trying to use Information Gain to select features when classifying text with a Support Vector Machine.
For each word in our training data, we computed its information gain. Then, we should keep only the words with an IG superior to some threshold.
When reading the litterature about this I did not find a clear explanation about how we should select this thresold.
Given a training corpus, for each unique term we computed the information gain, and removed from the feature space those terms whose information gain was less than some predetermined threshold.
For each dataset we selected the subset of features with non-zero information gain.
When training our SVM with words having IG > 0 as suggested in this paper, the results are worse than without feature-selection.
When using IG > 0.01 the results become better.
Is there a recommended way for selecting this IG threshold ?
Should we perform a grid-search (with Training and Cross Validation data set) to determine the best IG threshold ? In which case this threshold becomes a meta-parameter such as the C of the SVM model.
Should we stick to the simple IG > 0 even if our generalization results are worse ?