Skip to main content
Cross Validated

Return to Answer

added 111 characters in body
Source Link
Isabella Ghement
Isabella Ghement
  • 20.9k
  • 2
  • 36
  • 60

This is an interesting problem from a pedagogical viewpoint.

In the context of your 3 t-tests example, if you were to use a Bonferonni correction for multiplicity, you would want to perform the first test at a significance level $\alpha_1$, the second test at a significance level $\alpha_2$ and the third test at the significance level $\alpha_3$, so that:

$\alpha_1 + \alpha_2 + \alpha_3 = \alpha$,

where $\alpha$ is the experimentwise error.

You could choose $\alpha_1 = \alpha/3$, $\alpha_2 = \alpha/3$ and $\alpha_3 = \alpha/3$. With this choice, $\alpha$ is equally allocated among the 3 t-tests. (Unequal allocation of $\alpha$ is also possible.)

Let $p_1$ be the unadjusted p-value for the first t-test, $p_2$ be the unadjusted p-value for the second t-test and $p_3$ be the unadjusted p-value for the third t-test.

For each $i$, you can compare $p_i$ against $\alpha_i$ to decide whether or not you can reject the corresponding null hypothesis $H_{0i}$ at the significance level $\alpha_i$. If $p_i \leq \alpha_i$, then $H_{0i}$ can be rejected at the significance level $\alpha_i$. Since $\alpha_i = \alpha/3$, $H_{0i}$ can be rejected when $3 \times p_i \leq \alpha$.

  The Bonferroni-adjusted p-value for the $i$-th test is therefore $3 \times p_i$.

This paper on Adjusted P-Values for Simultanous Inference by S. Paul Wright (Biometrics 48, 1005-1013, December 1992) will give you a nice overview of other types of p-value adjustments you can make: http://www-stat.wharton.upenn.edu/~steele/Courses/956/Resource/MultipleComparision/Writght92.pdf.

This is an interesting problem from a pedagogical viewpoint.

In the context of your 3 t-tests example, if you were to use a Bonferonni correction for multiplicity, you would want to perform the first test at a significance level $\alpha_1$, the second test at a significance level $\alpha_2$ and the third test at the significance level $\alpha_3$, so that:

$\alpha_1 + \alpha_2 + \alpha_3 = \alpha$,

where $\alpha$ is the experimentwise error.

You could choose $\alpha_1 = \alpha/3$, $\alpha_2 = \alpha/3$ and $\alpha_3 = \alpha/3$. With this choice, $\alpha$ is equally allocated among the 3 t-tests. (Unequal allocation of $\alpha$ is also possible.)

Let $p_1$ be the unadjusted p-value for the first t-test, $p_2$ be the unadjusted p-value for the second t-test and $p_3$ be the unadjusted p-value for the third t-test.

For each $i$, you can compare $p_i$ against $\alpha_i$ to decide whether or not you can reject the corresponding null hypothesis $H_{0i}$ at the significance level $\alpha_i$. If $p_i \leq \alpha_i$, then $H_{0i}$ can be rejected at the significance level $\alpha_i$. Since $\alpha_i = \alpha/3$, $H_{0i}$ can be rejected when $3 \times p_i \leq \alpha$.

  The Bonferroni-adjusted p-value for the $i$-th test is therefore $3 \times p_i$.

This paper will give you a nice overview of other types of p-value adjustments you can make: http://www-stat.wharton.upenn.edu/~steele/Courses/956/Resource/MultipleComparision/Writght92.pdf.

This is an interesting problem from a pedagogical viewpoint.

In the context of your 3 t-tests example, if you were to use a Bonferonni correction for multiplicity, you would want to perform the first test at a significance level $\alpha_1$, the second test at a significance level $\alpha_2$ and the third test at the significance level $\alpha_3$, so that:

$\alpha_1 + \alpha_2 + \alpha_3 = \alpha$,

where $\alpha$ is the experimentwise error.

You could choose $\alpha_1 = \alpha/3$, $\alpha_2 = \alpha/3$ and $\alpha_3 = \alpha/3$. With this choice, $\alpha$ is equally allocated among the 3 t-tests. (Unequal allocation of $\alpha$ is also possible.)

Let $p_1$ be the unadjusted p-value for the first t-test, $p_2$ be the unadjusted p-value for the second t-test and $p_3$ be the unadjusted p-value for the third t-test.

For each $i$, you can compare $p_i$ against $\alpha_i$ to decide whether or not you can reject the corresponding null hypothesis $H_{0i}$ at the significance level $\alpha_i$. If $p_i \leq \alpha_i$, then $H_{0i}$ can be rejected at the significance level $\alpha_i$. Since $\alpha_i = \alpha/3$, $H_{0i}$ can be rejected when $3 \times p_i \leq \alpha$. The Bonferroni-adjusted p-value for the $i$-th test is therefore $3 \times p_i$.

This paper on Adjusted P-Values for Simultanous Inference by S. Paul Wright (Biometrics 48, 1005-1013, December 1992) will give you a nice overview of other types of p-value adjustments you can make: http://www-stat.wharton.upenn.edu/~steele/Courses/956/Resource/MultipleComparision/Writght92.pdf.

Source Link
Isabella Ghement
Isabella Ghement
  • 20.9k
  • 2
  • 36
  • 60

This is an interesting problem from a pedagogical viewpoint.

In the context of your 3 t-tests example, if you were to use a Bonferonni correction for multiplicity, you would want to perform the first test at a significance level $\alpha_1$, the second test at a significance level $\alpha_2$ and the third test at the significance level $\alpha_3$, so that:

$\alpha_1 + \alpha_2 + \alpha_3 = \alpha$,

where $\alpha$ is the experimentwise error.

You could choose $\alpha_1 = \alpha/3$, $\alpha_2 = \alpha/3$ and $\alpha_3 = \alpha/3$. With this choice, $\alpha$ is equally allocated among the 3 t-tests. (Unequal allocation of $\alpha$ is also possible.)

Let $p_1$ be the unadjusted p-value for the first t-test, $p_2$ be the unadjusted p-value for the second t-test and $p_3$ be the unadjusted p-value for the third t-test.

For each $i$, you can compare $p_i$ against $\alpha_i$ to decide whether or not you can reject the corresponding null hypothesis $H_{0i}$ at the significance level $\alpha_i$. If $p_i \leq \alpha_i$, then $H_{0i}$ can be rejected at the significance level $\alpha_i$. Since $\alpha_i = \alpha/3$, $H_{0i}$ can be rejected when $3 \times p_i \leq \alpha$.

The Bonferroni-adjusted p-value for the $i$-th test is therefore $3 \times p_i$.

This paper will give you a nice overview of other types of p-value adjustments you can make: http://www-stat.wharton.upenn.edu/~steele/Courses/956/Resource/MultipleComparision/Writght92.pdf.