# Why do we calculate the family-wise Type-I error rate by assuming statistical independence of pair-wise comparisons?

$$FWER = 1-\big(1-\alpha\big)^m$$

The above formula is typically used when calculating the $$FWER$$ where alpha is the criterion applied by the researcher to each test and $$m$$ is the number of tests.

However, in multiple places, I see that this only applies when the tests are statistically independent (i.e. comparing means of measurements, e.g., from groups of different/unrelated participants).

Doesn't this go against the idea that a family is: "all those experimental observations that could be analysed statistically by a global procedure’ (such as an omnibus test)" (Ludbrook, 1998). Such procedures test global null hypotheses (i.e., that all local null hypotheses are true), assuming that the observations are from the same population (e.g., having the same mean). In this sense, the individual test cannot by definition be statistically-independent because to some degree they are all evaluating the same null hypothesis.

Why would we then assume that tests are independent?

This makes no sense to me. It seems contradictory.

Ludbrook J. Multiple comparison procedures updated. Clin Exp Pharmacol Physiol. 1998;25:1032–1037.

• It’s more of a bound than an equality. Think of the equation you gave as the worst-case scenario.
– Dave
May 25, 2020 at 3:32
• Is there any reason why we would make such worst-case scenario assumption (i.e., of fully independent pair-wise comparisons)? I am asking because in other cases in statistics when we make assumptions we make best-case scenario assumptions (e.g., when computing ANOVA we assume normally-distributed DV values within-conditions with equal variances between conditions). I guess now my question is why we sometimes assumes the worst, while other time - the best? May 25, 2020 at 3:44

However, people do often use the Bonferroni bound $$\textrm{FWER}= 1-m\alpha$$, precisely because it makes no assumptions. The argument against the Bonferroni bound is that it's conservative. It is, but it's not very conservative for large $$m$$ and small $$\alpha$$ unless the correlation is truly extreme.