Calculate sample size for Fisher Exact Test I'm researching the role of sender reputation in remediating Android vulnerabilities using large-scale notification campaign. There are 2 senders with different reputations and 1 as a control (no notification sent). I am measuring the effect of these notifications by the senders and finding out how many vulnerabilities were fixed when they were notified. I'm having a problem in the study design as I can't seem to figure out the sample size needed for a meaningful analysis. This is the contingency table so far.
          Fixed   Not fixed  Total
Sender1                        21
Sender2                        21
Control                        22

Total                          64

I know I have to do it pairwise and then use Multitest corrections (I'll be using Holm-Bonferrroni for this), but the question is whether the analysis will be meaningful or not? For that I have to calculate sample size needed. How to do that?
(The question might be very basic but I don't have a background in stats so pls help)
 A: I will try to answer you question, please feel free to correct me if I understand you wrongly.
First, here is a sample size calculator for fishers exact test or use this package in R, and here an article. If you are interested in further explanation, I would suggest to google, fishers exact test is well described.
After this is said, to calculate sample size, you need prior knowledge. Sample size should NEVER be calculated based on the observations used for the actual analysis.
An alterative for using fishers exact test for calculating sample size is to use proportions.
Second you ask about correction for multiple testing. Unless you are looking at 1000 or more tests, I would never recommend correcting for multiple testing. Why? Because you simply increase type 2 errors when trying to decrease type 1 errors, and how can you make sure you do not want to perform a test later on the same data set? If you do so and are determined on correction you previous results are no longer valid but should be corrected for the new additional tests. Here is an article that comment on these problems.
I hope this may be of help
