1
$\begingroup$

I am following an SPSS tutorial for Kruskall-wallis test. In the example give, we are comparing the driving reaction time for 3 groups (each group contain 10 subjects) following a drink. 1st grp was given water, 2nd coffee, 3rd alcohol. Kruskall-wallis was significant. A Mann-whitney test was carried on each pair of grps. enter image description here

As multiple tests are being carried out, SPSS makes an adjustment to the p-value. The adjustment here is to multiply each Mann-Whitney p-value by the total number of Mann-Whitney tests being carried out (Bonferroni correction). But as I know, bonferroni correction should decrease the p value and its formula is p value/nb of tests. So the adj sig should be smaller than the sig. So the sig and adj sig columns should be reversed. Where am I mistaken?

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ It is not appropriate to run separate WIlcoxon tests after a K-W test because the Wilcoxon tests re-rank the variables resulting in non-transitivity e.g. A > B > C > A. More at hbiostat.org/doc/bbr.pdf which points out how its better to use the regression model generalization of the K-W and Wilcoxon tests, the proportional odds model. $\endgroup$
    – Frank Harrell
    Aug 16, 2020 at 11:12
  • 3
    $\begingroup$ @Ahmad Leaving aside the issue Frank raised, in general when accounnting for multiple testing, either the significance level should go down (while p-values remain the same) or p-values should be adjusted upward (while significance levels remain the same) $\endgroup$
    – Glen_b
    Aug 16, 2020 at 12:15

1 Answer 1

Reset to default
2
$\begingroup$

The Bonferroni correction adjusts the $\alpha$-level, the threshold required to declare a result significant. This makes it harder to reject the null hypothesis in order to prevent a true null from slipping throttle cracks just because you kept on trying to find a false null among many true nulls e.g. the XKCD on jelly beans.

Alternatively, the p-value could be adjusted up while keeping $\alpha$ where it was.

Either way, the goal is to make it harder to reject the null hypothesis.

Improve this answer
$\endgroup$
4
  • $\begingroup$ So in the picture attached, we adjusted up the p-value but kept the level of significance at 0.05. If we were to go with the Bonferroni correction, we would adjust α -level for the water-coffee pair, for example, at 0.012. Right? $\endgroup$
    – Ahmad Dabboussi
    Aug 16, 2020 at 15:13
  • $\begingroup$ Where are you getting $0.012$? $\endgroup$
    – Dave
    Aug 17, 2020 at 2:18
  • $\begingroup$ For the first pair: 0.036/3= 0.012. Shouldn't the adjusted sig be lower than the sig? $\endgroup$
    – Ahmad Dabboussi
    Aug 17, 2020 at 9:27
  • $\begingroup$ I think that “sig” column is the p-value, so the appropriate arithmetic is to multiply by three, not divide. You’re trying to make rejection harder. $\endgroup$
    – Dave
    Aug 17, 2020 at 9:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.