I've been trying to perform a binary classification using an SVM classifier (scikit-learn's SVC with RBF kernel). I have a sample size of about 100, with about 70 features each. The features are of approximately the same order of magnitude in their raw form, and the values tend to be already distributed around the 0 (not always though). The distribution of two such features is shown in the histograms below.
I performed a Z-score transformation on all features, as I know this to be considered a good practice when working with multiple features in machine learning. The problem is that when I use the raw data, I always manage to get better accuracy than with the Z-scores (about 2-3%). Bear in mind that the parameters of the SVMs in each case are optimized using a grid-search, so I'm not using exactly identical classifiers.
Does this make sense, getting worse results with Z-scores? I would expect to get the same or better results. What could be the mathematical logic behind this?
To answer two frequent questions from comments and answers:
- My classes are indeed distributed equally (exactly 50%/50%)
- I also use measures other than accuracy (AUC, F1, etc.), but having worked on this project for some time, accuracy correlates well with what I need.