I apply epsilon Support Vector Machine (e-SVM) to a regression problem via Weka. I have about 6000 features and 2000 samples. I order the feature respect to minimal-redundancy-maximal-relevance criterion and feed SVM with different number of feature size.
I think that the performance for more informative feature sets after a proper feature selection should be higher than total features. However, the performance of total feature size (6000 dimensions) is about 0.49 (Pearson correlation metric) and the maximum correlation for feature size between 1 to 200 is 0.45 which is worse.
My question is that, does e-SVM do kinds of feature selection or dimension reduction?! As I know from basic machine learning, when the feature dimension is too high compare to sample size, we have sparsity problem and it leads to not proper generalize model from train to test data!