I try to classify a brainstate (binary problem) on fMRI-data using a SVM (scikit-learn, which wrapps libsvm). Also I use clusters arround local maxima in group-level TMaps as mask for the subject scanns, leading to roughly p=1500 features. Given between 85 and 90 examples, using a 5 fold cross validation I have a trainingset size arround n=70. I would now expect the SVM to overfit, and to have nearly 100% accuracy on the training set, and basically nothing on the test set (at least when unregulated). But what actually happens is that in the training set, as well as in the test set, the classifier just always predicts the label that was more likely in the training set. I tried a linear SVM, a gaussian SVM, l1 and l2 regularization, and to provoke the overfitting a C of $1^{-25}$. But everything lead to the same result. Except for one case. To test for errors, and the capeablity of the SVM to extract extremly sparse solutions i replaced the first entry of the Featurevector with the target class, which led to 100% classification accuracy for the linear-kernel with l1 and l2, but not for the Gaussian kernel, for which the SVM reached perfect classification in 10% of cases, and in the other 90% it behaved like above.
So now my 3 questions:
- Why does the SVM not overfit, not even with a C of $1^{-25}$?
- Why does the Gaussian-kernel find the correct solution only in 10% of cases while the linear-kernel does it every time? (My understanding is, that the linear-kernel implies no Feature-space transformation, while the gaussian one does. And as far as I understand it, that means, that the linear SVM works in a 1500-dimensional space, while the gaussian one should work in one whichs dimension is at most equal to the number of training-examples (70 in this case) so it should be (a lot) easier for the gaussian-kernel to find the solution than for the linear-kernel
- Do I have to conclude now, that my Feature vectors simply do not contain the Information, that allows to predict the labels?
