What will happen to my results if I use Fisher's exact test with unfixed marginal totals?

Question

I've been trying to get my head around the controversy that surrounds Fisher's exact test.

I often use the test in the something like the following way:

1. A survey asks respondents two questions: "Are you blonde?" and "Do you like skateboarding"?
2. After a month of collecting responses, we have 16 responses and can make the following contingency table:

skateboards	is_blonde	not_blonde
Yes	7	2
No	1	6

3. At this point I run a fisher's exact test, which gives me a p value that's less than 0.05, which I consider acceptable to accept the hypothesis that being blonde and doing skateboarding are not independent.

If I've understood the literature rightly, then neither of my marginal totals were fixed by design: The respective numbers of respondents, skateboarders and blondes were all unknown to me prior to the closing of my survey. In practice, how will this affect my results?

I understand that Fisher's exact test is a conservative test. When marginal totals are unfixed, will it err towards failing to reject the null? I work in a relatively non-academic context, so a small loss of statistical power wouldn't be a problem.

Answer 1

First thing is that as you have a significant result, whatever the power was was enough to find it, so for these specific data I wouldn't worry about loss of power.

Assuming that the marginal totals are random and not fixed, in fact you'd need to assume to know their distribution in order to figure out the power. Obviously, making some assumptions about it, you can find out something by simulation, i.e., generate lots of data sets under a specific choice of the alternative (you can use several choices to get a more comprehensive picture, including assuming marginals to be fixed), and then you see what the power would be.

As your marginals are pretty close to uniform, I'd expect a somewhat weaker power indeed if there is a considerable probability to have them less balanced (assuming that the sample size is fixed although in principle this could be modelled as random as well). But that shouldn't affect your conclusions (which by the way I find badly worded - the term "accept" is usually applied to the null hypothesis and even there misleading, as statistical models are always idealisations and never literally true; also, as said in a comment, Fisher's test is two-sided and will not imply that deviation is in a specific direction. I'd rather say "there is evidence against skateboarding and being blond being independent").