0
$\begingroup$

For my thesis, I am comparing the importance of "comfort" (measured using a 6-point Likert scale) among four groups of people (participants who prefer traveling by plane, by train, by coach and by motor vehicle). However, I get different significance levels when doing the Kruskal-Wallis test and the Mann-Whitney U test.

In the Kruskal-Wallis test I get: chi-squared = 12.262, df = 3, p-value = 0.006537

When performing the Mann-Whitney U test (with "bonferroni" correction), however there is no statistical difference among the groups:

enter image description here

How should I interpret the data & is there another test that could give me better results than the two I performed?

May I note that the number of data sets highly varies between the 4 groups (n=272, n=68, n=9, n=140).

Improve this question
$\endgroup$
4
  • $\begingroup$ Bonferroni adjustments to guard against 'false discovery' are based on an inequality and are often quite conservative ('reluctant' to find significant differences). So it often happens that K-W finds significant differences, but Bonferroni-adjusted MWW tests do not. // In your case, the very small sample size (for coach?) results in very small power for MWW. // My guess is that your most extreme difference is btw Plane and Train (with sample sizes 272 and 140?). This may drive the KW result, and the unadjusted P-value btw Plane and Train for MWW must also be near 0.01. $\endgroup$
    – BruceET
    Oct 13, 2020 at 19:56
  • $\begingroup$ You have not shown much details, so one can only speculate what is going on. If you are free to share more details (e.g, summary statistics for the four levels), maybe one of us can give a more detailed explanation. $\endgroup$
    – BruceET
    Oct 13, 2020 at 19:59
  • $\begingroup$ The post hoc pairwise comparison test after the omnibus KW is called Dunn test. It is not just MW but a version of MW. $\endgroup$
    – ttnphns
    Oct 14, 2020 at 0:35
  • $\begingroup$ Mann-Whitney is an inappropriate test following rejection of the Kruskal-Wallis. $\endgroup$
    – Alexis
    Nov 13, 2021 at 19:57

1 Answer 1

Reset to default
1
$\begingroup$

Here are simulated data that may be sufficiently similar to yours to illustrate my Comments. Likert scores are simulated in R, using various vectors of preference for the four modes of transportation. If you use the same seed I used, you will get the same hypothetical data. [R converts these vectors to probabilities before use in sample.]

set.seed(2020)
plane = sample(1:6, 272, rep=T, p = c(1,2,2,3,3,3))
auto  = sample(1:6,  68, rep=T, p = c(1,2,3,3,2,1))
coach = sample(1:6,   9, rep=T, p = c(1,2,3,3,2,1))
train = sample(1:6, 140, rep=T, p = c(1,2,3,2,2,1))
all = c(plane,auto,coach,train)
gp = rep(1:4, c(272,68,9,140))

In the boxplot below, widths of boxes indicate varying sample sizes, and notches in the sides of the boxes are approximate nonparametric confidence intervals for group medians (calibrated so that two nonoverlapping intervals suggest a significant difference in location). [The sample size for Coach is too small for a meaningful CI.]

boxplot(all ~ gp, varwidth=T, col="skyblue2", notch=T, 
        names=c("Plane","Auto","Coach","Train"))

enter image description here

A Kruskal-Wallis test finds significant differences among the centers of the populations at the 5% level.

kruskal.test(all ~ gp)

        Kruskal-Wallis rank sum test

data:  all by gp
Kruskal-Wallis chi-squared = 8.6787, df = 3, p-value = 0.03388

However, a nonparametric Mann-Whitney-Wilcoxon (rank sum test) between Plane and Train give P-value about $0.012,$ but one version of the ad hoc Bonferroni P-value is about $0.07$ (not significant at 5%).

wilcox.test(plane, train)

        Wilcoxon rank sum test with continuity correction

data:  plane and train
W = 21863, p-value = 0.01212
alternative hypothesis: true location shift is not equal to 0

Moreover, MWW comparisons of Coach with other groups shows no significant difference (even without a Bonferroni adjustment), on account of the small sample size for Coach.

wilcox.test(plane, coach)

        Wilcoxon rank sum test with continuity correction

data:  plane and coach
W = 1318, p-value = 0.6913
alternative hypothesis: true location shift is not equal to 0

Note: In these circumstances, my personal opinion is that it it's OK to declare a significant difference between Plane and Train because the K-W test finds 'difference(s)', of which at least this difference must be one. But I would not feel comfortable finding differences involving Auto and Coach without Bonferroni (or some other) protection against 'false discovery'.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.