# Multiple Comparison Correct Group of Confidence Intervals

I have a number of data points (~10,000) compared against different confidence intervals to determine whether or not they are significantly elevated or not.

Now I would like to correct for multiple comparison testing, but the only way I know how to do this is by using the p-values from each comparison, which I don't have because they were compared to a confidence interval.

How can I correct for multiple comparison?

A common way to correct for $$m$$ multiple comparisons when it comes to hypothesis tests is the Bonferroni correction: by rejecting each null hypothesis $$H_i:i\in\{1,2,...,m\}$$ at a Type 1 error rate $$\alpha/m$$, the familywise error rate (FWER) is controlled at $$\leq \alpha$$.

The Bonferroni correction is derived from the Boole-Frechet inequalities, which relates the probabilities of multiple events to the probability of their intersection and the probability of their union. Given $$m$$ events $$E_i:i\in\{1,2,...,m\}$$,

$$1-m+\sum^m_{i=1}P(E_i) \leq P(\bigcap^m_{i=1}E_i) \leq \min(P(E_i):i\in\{1,2,...,m\}) \leq \max(P(E_i):i\in\{1,2,...,m\}) \leq P(\bigcup^m_{i=1}E_i) \leq \sum^m_{i=1}P(E_i)$$

Say the event $$E_i$$ is a Type 1 error for the $$i$$th test, then if all $$P(E_i)=\alpha/m$$, the FWER $$P(\bigcup^m_{i=1}E_i) \leq m\alpha/m=\alpha$$. It's that simple.

With multiple confidence intervals, the event $$E_i$$ is each interval. You're interested in controlling the probability that all confidence intervals contain their estimated parameter, say $$\geq 1-\alpha$$. In this case we're looking at the intersection probability, and assuming the individual confidences $$P(E_i)$$ are all equal: $$1-\alpha = 1-m+mP(E_i) \leq P(\bigcap^m_{i=1}E_i)$$. It follows that $$P(E_i)=1-\alpha/m$$.