I have a dataset for classification (binary - 1/0) that has around 4000 samples that I use to train the model (I'm using an SVM, if that's relevant). To check whether everything is working fine, I cross validate 5 folds. But I'm noticing that the cross validation accuracy that I get is the exact same fraction of the 0 labels in the set.
For instance, when I pick the last 800 samples for testing and train on the first 3200 samples, the accuracy of prediction on the last 800 samples is the exact same number as the fraction of samples having label = 0 in those 800 data points. Does this mean that the classifier is not taking the features into consideration at all? Or is this completely normal behavior and I'm missing something?