I have a classification problem (bioinformatics domain) where I have around 333 features. Currently, I am first selecting features (using importance feature of random forest) and then pushing the same through RBF kernel for SVM. On the results front, I have section of data which gives bad results on the classifier. There is feature rscindex, which separates the data into well classified data and incorrectly classified data,i.e data points with rscindex < 0.8 are performingly badly during classification stage. Please give me ideas on how to improve my results on this badly performing section of data. Will using Fisher kernel help (if so is there a library I can use) or is there a sampling technique that I should be looking into.? Please help. Thanks
- make sure your parameters are properly tuned (in the case of SVM you will probably have $C$ and $\gamma$ for the kernel)
- try without feature selection (SVM is fairly robust to useless features anyway).