I have a situation where I was planning to use post-hoc Fisher's exact tests to look for significant differences in the ratios of populations of cell subsets in response to treatments, but I'm concerned that this approach is violating Fisher's assumption of independence in my particular case. I'm wondering if this is indeed a violation of this assumption and if there are alternative approaches other than redesigning the experiment.
Consider a situation where you have two treatments, Tx_1 and Tx_2, each applied to a single dish of cells (one dish per treatment). In response to these treatments, the cells segregate into distinct populations (for which we have count data of course). I've been using Fisher's exact test to see if these treatments have distinct impacts on how the populations segregate, e.g.:
> ct Tx_1 Tx_2 Population_A 2 12 Population_B 13 2 Population_C 16 12 Population_D 3 3 > fisher.test(ct) Fisher's Exact Test for Count Data data: ct p-value = 0.0007053 alternative hypothesis: two.sided
In the example above, the there is a significant difference of treatment across the populations in general. However, I would now like to know -which- populations were different across the treatments (and which were not different). For instance, some populations of cells may have to exist regardless of treatment, but some outcomes will definitely be impacted by treatment. I had thought I could test this by doing post-hoc Fisher's tests for each outcome (followed by pvalue adjustment)), where I'm using the frequency of one populations and the total sample size not from that population in the contingency table, e.g.:
> ct.A Tx_1 Tx_2 Population_A 2 12 OtherPopulations 32 17 > fisher.test(ct.A) Fisher's Exact Test for Count Data data: ct.A p-value = 0.0016 alternative hypothesis: true odds ratio is not equal to 1 95 percent confidence interval: 0.00900654 0.48400271 sample estimates: odds ratio 0.09210876 > > ##Population B > ct.B <- matrix(c(ct["Population_B","Tx_1"], ct["Population_B","Tx_2"], + sum(ct[,"Tx_1"]) - ct["Population_B","Tx_1"], sum(ct[,"Tx_2"]) - ct["Population_B","Tx_2"]), nrow = 2, + dimnames = + list(c("Population_B", "OtherPopulations"), + c("Tx_1", "Tx_2"))) > ct.B Tx_1 Tx_2 Population_B 13 21 OtherPopulations 2 27 > fisher.test(ct.B) Fisher's Exact Test for Count Data data: ct.B p-value = 0.006307 alternative hypothesis: true odds ratio is not equal to 1 95 percent confidence interval: 1.571899 81.510429 sample estimates: odds ratio 8.098433
But my concern is that this approach is invalid because the the total number of cells in a treatment is fixed, so if one population increases in response to that treatment, then some other population has to decrease in response to that treatment to make up the difference. Is this violating Fisher's assumption of independence, or am I way overthinking this? If so, is the only alternative approach to go back and redo the experiment for additional dishes of cells and then use something like a T test to compare differences in populations as an effect of treatment, or are there other approaches that can leverage the data we have currently?