# Using a binomial approach to calculate significance of multiple significance tests

I have 10 data sets with normal distributions and carrying out significance tests on each one. With this many data sets, there is a good chance that some will be significant by chance. I am going to try the Bonferroni method to solve this problem but another approach that occurred to me is a binomial one.

i.e. if the p value for each individual significance is 0.01 then the probability of getting a significant result follows B(10, 0.01). You can then generate the probability of any number of significant results. Is this method "legit"? I can't see anything wrong with it and it has been mentioned in another thread but does not seem to be very common and woud like to know if there is an issue with using it.

If you're interested in some overall sense of significance, then yes you could combine independent tests with constant $\alpha$ using the binomial. *
* The reliance on whether individual tests achieve some particular $\alpha$ is somewhat arbitrary. To get some overall idea of signficiance, it would instead be more common to combine the p-values, for example via Fisher's method.
• e.g. (0.0627, 0.4855, 0.1789, 0.0931, 0.7950, 0.0836, 0.4894, 0.0973, 0.0914, 0.3796) has no p-values below 0.06, yet Fisher's method gives p=0.0304 – Glen_b -Reinstate Monica Aug 17 '14 at 0:08