Why are simultaneous confidence intervals wider than the normal ones?

Why are the simultaneous confidence intervals wider than the normal ones?

Aren't they made for securing that intervals equal amongst all the parameters, so separate intervals will be made for each parameter? Why does that make them wider?

• In short, they are wider because they have to secure a joint confidence level. – JohnK Dec 27 '15 at 19:20
• So when you say joint you mean that µ = [$µ_0$,$µ_1$,$µ_2$] all have the same CI (95% ) , thus you create create confidence interval vector, defining the region for each variable, or you just make an overall Confidence region that securs that CI - 95% at least is fulfilled.. because that would make sense to why they are wider? A joint being = $µ_1$ and so on.. – Bob Burt Dec 27 '15 at 19:24
• maybe a clarification of what the joint confidence is could be useful – Bob Burt Dec 27 '15 at 19:40

Consider two parameters and assume for simplicity that they are estimated independently. Say the sample mean and variance of a normal population which are estimates for the respective population quantities and are known to be indepedent.

From basic statistics we know how to construct confidence intervals for $\mu$ and $\sigma^2$ based on our estimates. Since this is a normal distribution, we have two beautiful pivotal quantities, namely

$$\frac{\bar{X}-\mu}{\sigma/\sqrt{n}} \sim N(0,1)\quad \text{and} \quad \frac{\left(n-1 \right) S^2}{\sigma^2} \sim \chi^2_{n-1}$$

Based on these distributions we can now find quantiles such that

$$P \left[ -z_{0.975} < \frac{\bar{X}-\mu}{\sigma/\sqrt{n}} < z_{0.975} \right] =0.95 \quad \text{and} \quad P\left[ \chi^2_{0.025}<\frac{\left(n-1 \right) S^2}{\sigma^2} < \chi^2 _{0.975} \right] = 0.95$$

Easy enough but there is a logical fallacy in these intervals, which you might have picked up. In the first interval I have assumed that $\sigma$ is known and this is clearly not the case, otherwise we wouldn't need the second interval!

What we can do to resolve this contradiction is use the four inequalities to determine a confidence region for both $\mu$ and $\sigma^2$. Thus we will now be concerned with the probability

\begin{align} \label{eq:1} P\left[-z_{0.975} < \frac{\bar{X}-\mu}{\sigma/\sqrt{n}} < z_{0.975} \ , \ \chi^2_{0.025} < \frac{\left(n-1 \right) S^2}{\sigma^2} < \chi^2 _{0.975}\right] \tag{1} \end{align}

With a slight reformulation of the above inequalities it is possible to determine the boundaries of $\mu$ and $\sigma^2$. Here is how the resulting region looks like

Pay no attention to the labels in the graph, these are just minor differences in the notation of the quantiles. You can now project onto the coordinate axes to obtain a confidence interval for both $\mu$ and $\sigma^2$. But we still do not know the level of this confidence region.

We have now arrived at the crux of the matter. My question to you is exactly that : what is the confidence level of this region? Surely we have used two 95% intervals so at first one might think that we are still at the 95% level. A moment of reflection will reveal however that this is incorrect.

In this simplified setting the confidence level can be computed exactly using the independence $\bar{X}$ and $S^2$. Take a look at equation (\ref{eq:1}) and recall that for two independent events A and B, $P\left[ A \cap B \right] = P[A] P[B]$. Thus

$$P\left[z_{0.975} < \frac{\bar{X}-\mu}{\sigma/\sqrt{n}} < z_{0.975} \ , \ \chi^2_{0.025} < \frac{\left(n-1 \right) S^2}{\sigma^2} < \chi^2 _{0.975}\right] = 0.95^2 = 0.9025$$

And we have found out the hard way that two 95 % intervals do not mean that the joint confidence level, i.e. the confidence level of equation (\ref{eq:1}), is 0.95. By Bonferroni's inequality we can establish a lower bound - which is nearly attained in this case - but other than that not much can be said for the general case of non-independence.

In view of this, it is often preferable to take a conservative approach and specify simultaneous confidence intervals that have at least 0.95 coverage. Indeed, the concept of exact confidence interval no longer exists in simultaneous interval estimation. Tukey's, Scheffe's and even Bonferroni's procedures are all conservative in nature. For instance, in ANOVA when you want to test contrasts you will be able to build intervals that are at least of 0.95 level but you can never be sure what their exact level is.

Of course, the intervals then are much wider than in the individual cases. It appears that we cannot do better, however, so better be safe than sorry.

Hope this helps.

• Thanks for reply.. It really put things in perspective, and feel like an idiot for causing so much attention to my stupid problem.. (I hate my brain...) – Bob Burt Dec 27 '15 at 22:05
• So.. I think I get you point.. And it computation wise make sense but (and I fell like an idiot for writing this) my issue concerns multivariate stats. So as the P- variables has individually and CI of 95 % so picking them individually shouldn't affect the other variables.. I think i understand for you univariate case.. does that mean the for an multivariate case that the probability would be = $0.95^p$ As the case is valid for univariate and multivariate is just an P times univariate distributions.. – Bob Burt Dec 27 '15 at 22:05
• The probability of coverage is only $0.95^p$ for independent estimators. This is hardly ever the case and my example was an exception. This is also true in the multivariate setting and if you rely solely on individual confidence intervals, the confidence level will not be what you want. In the $T^2$ setting you are intetested in, this is the reason we build our confidence intervals from the ellipsoid. – JohnK Dec 27 '15 at 22:12
• Thats hardly the case? - would you elaborate.. – Bob Burt Dec 27 '15 at 22:17
• So i think i get it... But again.. How much wider, or when do determine that it is wide enough? – Bob Burt Dec 27 '15 at 22:17

Consider simultaneously estimating two parameters. We want to guarantee that, under repeated sampling, for some confidence level c and significance level $\alpha$, $(c = 1-\alpha )$% of our confidences interval pairs (one on each parameter) would both include the parameter.

Without loss of generality, let $\alpha$ = 0.05, c = 0.95. Let's construct single individual confidence intervals on each of them. The probability of each of these intervals containing their parameter is 0.95 under repeated sampling. Assuming independence, the probability of them both including their parameters under repeated sampling is 0.95*0.95 = 0.9025 < 0.95. If we wanted them both to include the true parameter values under repeated sampling 0.95 of the time, we would need $\alpha = \sqrt{0.95} = 0.975...$, which will obviously be a bigger interval on each parameter.

We will consider two levels of confidence here (we will assume that all the parameters we are measuring are the same for simplicity, so pretend we are simultaneous estimating the mean of two normal distributions in this discussion): one overall confidence level ($c_1$), and one confidence level for each parameter ($c_2$). $c_1$ is the long term probability of *every single one * of our confidence intervals successfully covering their respective parameters under repeated sampling, and is what we are directly interested in. If I were to say: "I want a 95% CI on these two parameters", the 95% = 0.95 corresponds to $c_1$. Once we have $c_1$, it falls to us to calculate $c_2$, which in our example here with two independent parameters is about 0.975. $c_2$ represents the "confidence level" we will have on each parameter. It has to be greater than if we were just estimating that parameter alone, and so will lead to a greater interval.

As the number of parameters to simultaneously estimate increases, the confidence of getting each individual one right will also have to increase for static overall confidence, because as we have more parameters to estimate, we have more opportunities for failure under repeated sampling.

For example, consider estimating a million parameters. If we give them all 0.95 individual confidence intervals, there's no way we would be say that the set of all confidence intervals here will include the true parameters under repeated sampling 95% of the time under repeated sampling.

• hmm.. but in what sense does it get wider.. I mean.. As far I understood it. I would just compute the CI for each individual parameter for instance imgur.com/bLFe75n if $x_1$ has the lowest lower bound and $x_p$ highest upper bound. and thereby you could set the limit for CI. If that was the case i would understand the with becomes a bit wider.. As most of the $x_i$ would entail a value close to $\bar{x}$, and doing it stated here will set the limits at the extremes... Or so i think.. – Bob Burt Dec 27 '15 at 20:32
• I added paragraph 3, let me know if it helps. Calculating a CI for each individual parameter is different from a simultaneous CI. So in your example image, each critical value (denoted F sub alpha... in your image) gets bigger as you add parameters to estimate as I demonstrated, increasing the width of the confidence interval. – John Madden Dec 27 '15 at 20:34
• How is this differently computed.. I think i get the concept that the simultaneous sampling with an simultaneous confidence interval does secure things. – Bob Burt Dec 27 '15 at 20:40
• What might be a little easier to see the point is if you don't assume independence and use a Bonferroni type correction: Then to get joint (simultaneous) 95% CI's, you can use individual 97.5 CI's for each parameter. But the individual 97.5 CI's will be larger than individual 95% CI's. (Note that there is a choice here: You could decide to use individual 96% CI for one parameter and individual 99% CI for the other, to get joint 95% CI's. But the resulting CI's will still be larger than individual 95% CIs) – Martha Dec 27 '15 at 21:01
• I think you've got it, as long as you're tracking that the probability x refers to each $\bar{x}_i$ individually in Case A. To put it another way, Case B says P(all $\mu_i$ in interval) = P($\mu_1$ in interval)*P($\mu_2$ in interval)*P($\mu_3$ in interval) (assuming independence again). The confidence level we control in Case B is the P(all $\mu_i$ in interval), and so the confidence levels on the right side of the equation must be bigger (because they are less than 1 and multiplying), so the intervals become wider for each individual coefficient (this has all described Case B) – John Madden Dec 27 '15 at 21:59