Skip to main content
edited body
Source Link
COOLSerdash
  • 31.2k
  • 10
  • 104
  • 157

As you correctly point out, the issue here is largely a matter of dealing with the problem of multiple comparisons. In the case of nested tests, this matter is complicated by the fact that the hypotheses for the tests have direct logical implications to each other, so you are right to think that a standard application of Bonferonni'sBonferroni's method would be problematic.

The method that is usually applied here is to first perform an over-arching test to see if there is any evidence of a difference across either subgroup. That test is not listed in your post, but it would test the hypotheses:

$$H_0: \boldsymbol{\mu}_A = \boldsymbol{\mu}_B \quad \quad \quad H_A: \boldsymbol{\mu}_A \neq \boldsymbol{\mu}_B,$$

where these vector parameters each contain the mean parameters for both subgroups. If there is no evidence of a difference then that ends the matter and the smaller tests are not performed. If there is evidence of a difference then we may then proceed to do the more specific tests 1-2. In a regression context, the overall test would be done using an F-test and the smaller tests would be done using t-tests.

As you correctly point out, the issue here is largely a matter of dealing with the problem of multiple comparisons. In the case of nested tests, this matter is complicated by the fact that the hypotheses for the tests have direct logical implications to each other, so you are right to think that a standard application of Bonferonni's method would be problematic.

The method that is usually applied here is to first perform an over-arching test to see if there is any evidence of a difference across either subgroup. That test is not listed in your post, but it would test the hypotheses:

$$H_0: \boldsymbol{\mu}_A = \boldsymbol{\mu}_B \quad \quad \quad H_A: \boldsymbol{\mu}_A \neq \boldsymbol{\mu}_B,$$

where these vector parameters each contain the mean parameters for both subgroups. If there is no evidence of a difference then that ends the matter and the smaller tests are not performed. If there is evidence of a difference then we may then proceed to do the more specific tests 1-2. In a regression context, the overall test would be done using an F-test and the smaller tests would be done using t-tests.

As you correctly point out, the issue here is largely a matter of dealing with the problem of multiple comparisons. In the case of nested tests, this matter is complicated by the fact that the hypotheses for the tests have direct logical implications to each other, so you are right to think that a standard application of Bonferroni's method would be problematic.

The method that is usually applied here is to first perform an over-arching test to see if there is any evidence of a difference across either subgroup. That test is not listed in your post, but it would test the hypotheses:

$$H_0: \boldsymbol{\mu}_A = \boldsymbol{\mu}_B \quad \quad \quad H_A: \boldsymbol{\mu}_A \neq \boldsymbol{\mu}_B,$$

where these vector parameters each contain the mean parameters for both subgroups. If there is no evidence of a difference then that ends the matter and the smaller tests are not performed. If there is evidence of a difference then we may then proceed to do the more specific tests 1-2. In a regression context, the overall test would be done using an F-test and the smaller tests would be done using t-tests.

Source Link
Ben
  • 132.9k
  • 7
  • 255
  • 588

As you correctly point out, the issue here is largely a matter of dealing with the problem of multiple comparisons. In the case of nested tests, this matter is complicated by the fact that the hypotheses for the tests have direct logical implications to each other, so you are right to think that a standard application of Bonferonni's method would be problematic.

The method that is usually applied here is to first perform an over-arching test to see if there is any evidence of a difference across either subgroup. That test is not listed in your post, but it would test the hypotheses:

$$H_0: \boldsymbol{\mu}_A = \boldsymbol{\mu}_B \quad \quad \quad H_A: \boldsymbol{\mu}_A \neq \boldsymbol{\mu}_B,$$

where these vector parameters each contain the mean parameters for both subgroups. If there is no evidence of a difference then that ends the matter and the smaller tests are not performed. If there is evidence of a difference then we may then proceed to do the more specific tests 1-2. In a regression context, the overall test would be done using an F-test and the smaller tests would be done using t-tests.