6
$\begingroup$

I have a bunch of variables organized into 10 different levels of a grouping factor. I'm doing some ANCOVA on particular variables and also plotting the data using boxplots. I'd like to add 84% confidence intervals to all the groups (since non-overlapping 84% CIs indicate a significant difference at alpha .05 - at least for two groups). I can do all this quite easily in R.

My question is - should I be applying a "family-wise" 84% CIs to all the groups? In other words, just as one would devalue an alpha level by the number of groups to obtain a family-wise alpha, should I inflate the CI a reciprocal amount to achieve a family-wise interval? This seems reasonable to me, but I haven't seen this discussed in the literature.

If alpha were CI were interchangeable for two or more groups the the family-wise 84% CI would be 99.5%, but i've read that alpha and CI are only interchangable for 1-sample situations. If this is the case, how would I go about calculating the family-wise confidence intervals for 10 (or any number) groups?

Any advice would be welcome.

best,

Steve

Improve this question
$\endgroup$
1
  • $\begingroup$ Articles directly about this would be hard to find... perhaps look up least significant difference scores... which is equivalent to what you ware trying to do. $\endgroup$
    – John
    Commented Aug 21, 2010 at 2:09

2 Answers 2

Reset to default
3
$\begingroup$

It sounds a reasonable solution if this is what important for you to present in the plot.

What this will give you (besides many questions, in case you are working with people who like statistics less then you), is a CI that is applicable to your situation which requires correction for multiple hypothesis.

What this won't give you, is the ability to compare difference between groups based on the CI.

Regarding the computation of the CI, you could use the p.adjust with something like simes which will still keep your FWE (family wise error), but will give you a wider window.

As to why you didn't find people writing about this, that is a good question, I don't know.

Improve this answer
$\endgroup$
1
$\begingroup$

For nearly equal sample sizes you can translate Tukey's HSD (Google it) into a set of individual CIs. For unequal sample sizes your approach may be doomed, because all pairwise comparisons cannot be reduced to pairwise comparisons of intervals: check out the literature on the Tukey-Kramer Method for details. (I know Stata and SAS both do these computations; contributed package DTK does it in R.)

Improve this answer
$\endgroup$
1
  • $\begingroup$ I know about Tukey's HSD. I think I was unclear in my question. While i'm doing ANCOVA, i'm not trying to generate CIs from post-hoc pairwise comparisons. I'm just generating them by bootstrapping the samples in each level of the factor (also tried using the t-distribution). The CIs are for the mean of each individual group (factor level). I agree with Tal's comment that this does not allow differences between groups to be assessed. It's easy to do a Bonferroni correction on these CIs, is it possible to do the Hochberg (or other) method? (p.adjust requires a vector of p values I don't have). $\endgroup$
    – user1029
    Commented Aug 21, 2010 at 23:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.