Simple question on probability computation for multiple comparisons

Question

In class, we illustrated the multiple comparisons problem through a simple example (no, not the xkcd jelly beans comic). Essentially, with $N$ independent tests each at level $\alpha$, we said the probability of rejecting at least one true null is $1-(1-\alpha)^N$ which tends to 1 as $N$ gets large for fixed $\alpha$.

My question: doesn't this computation assume all $N$ nulls are correct? Aren't we interested in the probability of rejecting at least one true null, regardless of which ones are correct?

Update: On further thought, I think it was probably just an illustrative example under highly refined conditions e.g. independence and all true nulls. But I gather that methods to control familywise error rate such as Bonferroni and Holm do so under arbitrary dependence and under any number of true nulls. Any other thoughts are welcome!

Answer 1

This does assume that the null hypotheses are true. It tells you about the probability of false positives. For false negatives, you would need to assume how many hypotheses are false. Here you ask “assuming that all null hypotheses are true, what is the probability of a bogus finding?”

As for the assumption of independence of the null distribution, this is about a scenario where you do hypothesis tests independently. If different hypotheses depend on each other it's questionable if you should throw independent hypothesis tests at them.