I am trying to analyse the following table:
5 5 5 5 4 5 5 5 5
3 4 3 3 4 4 3 5 5
Each number is a success score out of 5 attempts, so we can assume binomial (p, 5), for unknown p. Each column pertains to one person. The bottom row contains the "pre" test results and the top row is the "post" test results.
At a quick glance, every item in the top row is at least as large as the corresponding item in the bottom row, so by a quick sign test, the hypothesis that there is a difference pre and post should be significant.
If I do a paired t-test on these data (holding my nose, because each item is binomial (p,5)) I get a significant result. If I put the data into a 2 x 2 contingency table (pre post x right wrong) and ignore the matching, I get a significant result.
But when I do a contingency test of the table above, testing for independence conditional on the margins, the result is non-significant (p = 0.97). I used fisher.test in R, the Fisher exact test.
So what is the best way to test the hypothesis that there is a significant difference, pre and post, taking matching into account? And why does the analysis of independence not apply here?