I asked 200 survey participants the same sequence of eight multiple-choice questions with four answer options (A, B, C and D). Here are the results:
- All A's: 58
- All B's: 1
- All C's: 2
- All D's: 0
- Mixtures of A's, B's, C's and D's: 139
I want to work out whether these results are statistically significant, which I take to mean whether the probability that they occurred randomly is less than 0.001. I understand the expected value for each combination of answers – all A's, all B's, all C's, all D's, and each mixture of A's, B's, C's and D's – to be 200/(4^8), which is 0.003051758.
So here's the problem. I've read that a Chi-squared test requires all of the expected values to be greater than five, and in this case none of them is greater than five. I've also read that none of the observed values can be zero, and in this case one of them is zero. I've seen something about artificially combining categories to bring all the expected and observed values above five and zero respectively, but I don't understand how this can be done without artificially affecting the p-value. Finally, I've read a few things about Fisher's exact test, but all of them seem to suggest that I'd be allowed only a few rows of values, whereas in this case I have 65,536 (i.e. 4^8).
What is the most appropriate method in this circumstance?