3
$\begingroup$

I have been reading Brad Efron's book Computer Age Statistical Inference. Section 20.1 talks about simultaneous confidence intervals. If I understand the idea, it basically means that if variable A has 95% coverage and variable B has 95% coverage then the probability $P(A \cap B) = .95^2 = .81$.

Methods like Scheffe's adjustment extend the intervals so that all variables together have 95% coverage. In other words, using the above example, $P(A \cap B) = .95$.

Now that I am aware of this idea I have the following questions:

  1. Why is it not standard practice in all modern regression problems?
  2. Are "regular confidence intervals", those which are not stretched out until they are simultaneous, mostly invalid?
  3. If A and B are colinear variables does that mean that you don't have to stretch their regular intervals out as much as you might if they were completely independent?
$\endgroup$
2
$\begingroup$

Your calculation above works if A and B are independent, if A and B are not independent then the probability of both intervals containing the true combined parameters may be higher than or lower than the 0.81.

The individual (non-simultaneous, non-adjusted) confidence intervals answer a different question than the simultaneous ones, and both questions can be of interest at different times.

If the non adjusted C.I. does not include 0, but the adjusted or simultaneous one does include 0, how are you going to interpret that?

What all things do you include in your simultaneous intervals? should you adjust for the intercept? variables that could have been in the current model but you left out? If my main research question is about the relationship between x and y, but I include other variables to adjust for possibly lurking/confounding but I don't care about the "significance" of those other variables, should I include them in the adjustments?

My best guess for your question 1 is simplicity. The simple intervals are much simpler to compute (computers can generate reasonable ones automatically) but there are many ways to compute adjustments (Sheffe is just one) and deciding on which to use (and how to use it) depends on context, thought, and understanding that a computer should not be trusted to do automatically.

For 2. Those confidence intervals are valid for the question that they answer, it takes thought to determine if that is the question of interest or if some other routine needs to be used to give an appropriate answer to the question of interest.

For 3, the amount of adjustment depends on how they are related, being colinear is part of the relationship. This is another reason for not auto-adjusting everything, canned methods may get the adjustments wrong while giving people a false belief that everything has been properly adjusted.

$\endgroup$
  • $\begingroup$ as a follow up where are simultaneous intervals most used or, in theory, where would they be most effective? $\endgroup$ – Alex Nov 26 '19 at 23:51
  • 1
    $\begingroup$ @Alex, I don't know all the cases, but when doing an exploratory study and looking at all pairwise differences the simultaneous intervals are important (though these days I am leaning more towards a Bayesian approach here that would shrink all the means towards each other instead of just increasing the standard error). $\endgroup$ – Greg Snow Nov 27 '19 at 18:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.