I've been trying to get my head around the controversy that surrounds Fisher's exact test.
I often use the test in the something like the following way:
- A survey asks respondents two questions: "Are you blonde?" and "Do you like skateboarding"?
- After a month of collecting responses, we have 16 responses and can make the following contingency table:
- At this point I run a fisher's exact test, which gives me a p value that's less than 0.05, which I consider acceptable to accept the hypothesis that being blonde and doing skateboarding are not independent.
If I've understood the literature rightly, then neither of my marginal totals were fixed by design: The respective numbers of respondents, skateboarders and blondes were all unknown to me prior to the closing of my survey. In practice, how will this affect my results?
I understand that Fisher's exact test is a conservative test. When marginal totals are unfixed, will it err towards failing to reject the null? I work in a relatively non-academic context, so a small loss of statistical power wouldn't be a problem.