I have a dataset for classification (binary - 1/0) that has around 4000 samples that I use to train the model (I'm using an SVM, if that's relevant). To check whether everything is working fine, I cross validate 5 folds. But I'm noticing that the cross validation accuracy that I get is the exact same fraction of the 0 labels in the set.

For instance, when I pick the last 800 samples for testing and train on the first 3200 samples, the accuracy of prediction on the last 800 samples is the exact same number as the fraction of samples having label = 0 in those 800 data points. Does this mean that the classifier is not taking the features into consideration at all? Or is this completely normal behavior and I'm missing something?

  • 1
    $\begingroup$ So basically what you're saying is that your SVM classifier is classifying all testing data-set as 0. Is your data-set balanced (#of 1s is approximately equals to #of 0s)? $\endgroup$ Sep 2, 2012 at 20:28
  • $\begingroup$ @RoronoaZoro I wouldn't call it balanced. Around 70% are 1s and the rest 0s. $\endgroup$
    – Siddhant
    Sep 3, 2012 at 19:40
  • $\begingroup$ That's odd. I would imagine it's the other way around. So you're accuracy (# correctly classified / # all instances) would be really low then... Since you have more 1s... The bad thing about SVM is that there are many parameters to tweak, if you're using Libsvm library, you should play with the parameter -w (the weight)... Basically you increase the weight/penalty of classifying 1s as 0s. $\endgroup$ Sep 3, 2012 at 20:04

1 Answer 1


This sounds as though your classifier is assigning all cases to zero (classifying all cases into class labelled by 0). I suggest calculating the sensitivity for each class, i.e. the proportion of each class correctly classified, in order to check if this is the case. If you have unbalanced classes, try minimizing something other than overall classification error, such as (1-ROC area) or mean classification error across classes.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.