I have two differing-length sequences of binary values. In each sequence, each value corresponds to an observation which is either right or wrong.

I'm looking for a statistical test that addresses the question "are there significantly more right answers in one sequence?"

Which test is most appropriate?

I'm looking at Fisher's chi-squared test and Fisher's exact test, but I'm not exactly sure how to use each one.

EDIT: More details The study consisted of a 2-alternative forced choice recognition task. Two videos were displayed, then a third. Subjects had to match the third video with one of the first two, and the binary result collected was set to 1 if they were correct.

Trials were in random order, so shuffling the response vectors has no effect.



  • 3
    $\begingroup$ Does the sequence part of your question matter? $\endgroup$
    – John
    Nov 30, 2012 at 15:57
  • $\begingroup$ If you are determining right from wrong from continuous variables, you may be better served in using errors between your observed value and "actual" value. $\endgroup$
    – Nick Adams
    Nov 30, 2012 at 18:40

1 Answer 1


If your question is whether the proportion of correct answers depends on which sequence they come from, and the order or sequencing is irrelevant let's just call sequence "condition" then an appropriate test here is a chi-square test. If you have R installed sample code is below. I've named your conditions A and B.

dat <- matrix( c(numCorrectA, numWrongA, numCorrectB, numWrongB), ncol = 2 )

The component "numCorrectA, numWrongA, numCorrectB, numWrongB" should be replaced with the actual number of correct and incorrect items in conditions A and B.

Please Google chi-square test and read the wikipedia page for requirements, meaning, etc. before using the test. For example, you should make sure none of the numbers you're entering above is less than 5. In addition, it's a rather weak statistical evaluation from a theoretical standpoint should the sequences be very long. Trivial differences in them will come out as passing the test. Also, the individual samples should be independent. If they are not then update your question with more details of what exactly is the design of the study. It's possible that a comparison of confidence intervals from the binomial approximation to normal is best. Or, perhaps you need a McNemar's test or a multi-level logistic regression. or something else. It's too hard to tell from what you've provided what the best answer is.

  • $\begingroup$ Thanks very much for your input - I will read up thoroughly on this test. $\endgroup$ Dec 3, 2012 at 10:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.