# Multiple testing problem - can these test results be caused by chance?

I conduct 30 independent significance tests on $p<.05$. I observe that in $X=3$ cases the test is rejected. Can this event be caused by chance due to a multiple testing problem?

I understand that $X$ is binomially distributed, with expectation $E(X)=np=1.5$. What is the null hypothesis I should test, is it $H_0: p>.05?$. And how should this test be executed?

EDIT: I think it is $H_0: p<.05$, so we evaluate $P(X>3)=1-P(X <= 3)=.061$. So this is close, but one would not yet reject. From @whuber's comment there seems to be a more sophisticated approach that I need to consider.

• You seem to know the answer to your first question already :-). Consider a more powerful approach: study the distribution of all 30 p-values. Under the null, this will be a sampling distribution from a Uniform$(0,1)$ distribution (equivalently, their negative logarithms will have a $\Gamma(1)$ distribution). It would therefore be interesting to see the histogram of your p-values or--even better--an exponential probability plot. For more about this approach, please consult our posts related to Fisher's method. – whuber Apr 11 '14 at 15:57
• @whuber Could you point me to a post that gives hands on guidance on how to combine the p-values using Fisher's method? – tomka Apr 11 '14 at 16:07
• Just follow the link in my comment and read some of the first posts that show up. – whuber Apr 11 '14 at 16:49