I am calculating accuracy results from a single feature using LDA. I am very curious why a certain feature consistently performs at less than chance when calculating accuracy using a stratified 10-fold cross validation technique. I have an equally distributed binary class and performing multiple xvals results in an accuracy that fluctuates fairly tightly around 0.3.

This means the training data is consistently setting a boundary which when tested against is basically the opposite of what was trained on.

I am performing this stratified 10-fold xval 10-20 times and I would expect this to fluctuate around 0.5, but it is fairly static around the 0.3 to 0.4 area.

Thanks for the help!

  • 1
    $\begingroup$ What software are you using? Without knowing any specifics theres a good chance you have a bug in your code. $\endgroup$
    – BGreene
    Nov 8, 2012 at 9:17
  • $\begingroup$ The other option is a hard overfit or simply inconclusive data. $\endgroup$
    – user88
    Nov 9, 2012 at 0:53

1 Answer 1


So the problem appears to be a small number of samples and the fact that its a horrible feature. The p-value is close to 1, so the classes are pretty much drawn from the same distribution. Using synthetic data drawn from the same distribution with small N, I show there is a bias below 0.5 when performing multiple xvals that seems to asymptotically converge to 0.5 as N increases.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.