3
$\begingroup$

I am comparing 4 groups of samples using SPSS. The groups are 4 bathymetry groups (>0m, >-1m, >-2m, >-3m). Within each group are a number of samples n. Each sample has a recorded seagrass cover % per meter squared. n ranges between 6 and 65 for all 4 samples. Samples are not normally distributed. A kruskal wallis test is used to determine significant differences in the distribution of seagrass cover % per meter squared. When I compare all 4 groups (pairwise comparisons), my results show that one group is significantly different from the rest. However; when I remove that group from the test and redo the kruskal wallis test with the remaining 3 groups (that didn't show significant difference from each other in the previous test), one of the groups is now significantly different. If i remove that group, leaving me with 2 groups (which were not significantly different in the two previous test), and I do a Mann-Whitney U test, it says that the groups are now significantly different. Please explain what is going on here.

$\endgroup$
2
  • 3
    $\begingroup$ You are using the incorrect post hoc test. Mann-Whitney (a) does not use the same rankings as the K-S, so no surprise you are getting strange results, and (b) does not use the pooled variance assumed by the K-S null. Try Dunn's test or, even better, the Conover-Iman test. $\endgroup$
    – Alexis
    Apr 15, 2023 at 20:08
  • 1
    $\begingroup$ I'd add that the sequential procedure you describe -- find a group showing a difference then remove that group and look again -- without accounting for the impact of that sequential process on the properties of your inference is not likely just by happenstance to have properties you / your audience would find desirable. $\endgroup$
    – Glen_b
    Apr 16, 2023 at 3:24

1 Answer 1

6
$\begingroup$

You are using the incorrect post hoc test. Mann-Whitney (a) does not use the same rankings as the K-S, so no surprise you are getting strange results, and (b) does not use the pooled variance assumed by the K-S null. Try Dunn's test or, even better, the Conover-Iman test.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.