I train a SVM classifier using 36 features. If I use all the features, the train accuracy is about 0.96, the test accuracy is about 0.77. Then I change the number of features. The train accuracy drops while the test accuracy remains the same. Even if I just use one feature, the test accuracy is still about 0.77. I use cross validation, but the result does not change. I can not figure it out. So how does it come about?
It seems a bit suspicious that your constant accuracy number and the ratio of positives in your dataset are about the same, doesn't it?
It seems that your model is always predicting a positive label on the test set, though in some cases it's predicting negative on the train set.
Some possible remedies:
Try other hyperparameters. This sounds in particular like maybe your RBF kernel bandwidth is too small, if you're using that. Maybe try different kernels altogether.
Weight your training instances, so that the model cares more about getting the negatives right than it does about positives. Do this only if that's what you actually care about....
Maybe your features just aren't any good. Try getting better ones if you can, or perhaps weighting them differently or otherwise transforming them.