I know I can specify the adjustment for p-values (Bonferroni, Holm, etc). But I can set nothing for confidence intervals. How is that handled? I would like to set the Dunnett method for the confidence intervals, but it sets only the contrast. Neither in the documentation I cannot read about except that it is "multivariate" confidence intervals. How are the confidence intervals adjusted for multiplicity?


Let the ANOVA statistical model $$ y_{ij} = \gamma + \mu_i + \sigma\epsilon_{ij}, \quad i = 0, 1, \ldots, m, \quad j = 1, \ldots, n_i $$ where $\epsilon_{ij} \sim_{\textrm{iid}} \mathcal{N}(0,1)$.

Introduce the statistics $$ t_i = \frac{\bar{y_i} - \bar{y_0}}{\hat\sigma\sqrt{\frac{1}{n_i} + \frac{1}{n_0}}} $$ where $$ \hat\sigma^2 = \frac{\sum_{i=0}^m\sum_{j=1}^{n_i}{(y_{ij}-\bar{y}_i)^2}}{\nu} $$ with $\nu = \sum_{i=0}^m n_i - (m+1)$.

Then, under the null hypothesis $H_0\colon \{\mu_0=\mu_1=\cdots = \mu_m\}$, the random vector $\mathbf{t}=(t_1, \ldots, t_m)$ follows a multivariate Student distribution with $\nu$ degrees of freedom and scale matrix $\Sigma$ given by $$ \Sigma_{ij} = \begin{cases} 1 & \text{if } i = j \\ \sqrt{\frac{n_i}{n_i+n_0}}\sqrt{\frac{n_j}{n_j+n_0}} & \text{if } i \neq j \end{cases}. $$ Under $H_0$, let $q_p$ be the equicoordinate $p$-quantile of $\mathbf{t}$: $$ \Pr(t_1 \leq q_p, \ldots, t_m \leq q_p) = p. $$ Then the two-sided adjusted $100p\%$-confidence interval of $\mu_i - \mu_0$ is given by $$ (\bar{y_i} - \bar{y_0}) \pm q_{1-\frac{1-p}{2}}\times\frac{\hat\sigma}{\sqrt{\frac{1}{n_i}+\frac{1}{n_0}}}. $$

A bit more details on my blog.

  • $\begingroup$ This is excellent answer. Thank you. This should be really part of the documentation, like in SAS pages. Just one more question. So here I can see this gives me simultaneous confidence intervals. The same method seems to be used for Dunnett and Tukey. Am I right, that only the contrast is different (pairwise vs all-to-control), and the method of adjustment is the same? Is this comparable to Bonferroni? I am asking as my reviewer asked me to adjust p-values for each comparison using Bonferroni and Holm. He also asked me to adjust the CIs. I assume I could do that by correcting the alpha $\endgroup$ – pharmacist Feb 15 at 12:41
  • $\begingroup$ and display just the univariate Cis, calculated using the corrected alpha, according to the method I want, yes? $\endgroup$ – pharmacist Feb 15 at 12:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.