3
$\begingroup$

As part of the analysis of the data collected from a survey, I was carrying out Kruskal-Wallis Tests between some Likert scale questions and demographics such as firm size and job position. I am using this test instead of One Way Anova since the data is non-normal.

For example, testing the responses of the following statement vs Job position (manager or junior): I arrive late at work. Rated on a 5-point scale from never to always.

When reporting the results of the test (if result is significant) can I state something like:

Managers are more likely to arrive late than juniors (H=14.338, p<.01)

or is this the only way one can report the result:

A statistically significant difference (H=14.338, p<.01) exists between late arrivals at work by managers and juniors.

$\endgroup$
  • $\begingroup$ The Kruskal-Wallis test is used when the number of groups is three or more, otherwise it's reduced to Mann-Whitney test. However, you can interpret it like that. There's no need for post-hoc test if you have only two groups. $\endgroup$ – Germaniawerks Feb 14 '14 at 16:11
  • $\begingroup$ if there are more than 2 groups, would an error graph be sufficient to see which means differ significantly? $\endgroup$ – dissertationhelp Feb 14 '14 at 16:18
  • $\begingroup$ You need to apply Wilcoxon test with Bonferroni correction to the level of significance in order to compare which groups differ from which. $\endgroup$ – Germaniawerks Feb 14 '14 at 16:53
  • $\begingroup$ @dissertationhelp: I have just updated my answer with an important remark, please take a look. $\endgroup$ – amoeba Feb 14 '14 at 21:35
3
$\begingroup$

As @Germaniawerks remarked above, if you only have two groups (managers vs juniors) you should use ranksum (aka Mann-Whitney-Wilcoxon) test and there is no need for Kruskal-Wallis. If you have more than two groups, then Kruskal-Wallis will tell you if they are significantly different, but if you want to know which pairs are significantly different between each other, you need to do a post hoc comparison, e.g. ranksum test with Bonferroni correction.

Now answering specifically your question: I think your first formulation is completely acceptable.

But personally, I don't think it makes a lot of sense to report U statistic (in case of comparison between two groups, it should be U of Mann-Whitney, as explained above): few people have intuitive understanding of it, and this particular number (U=14.338) does not convey anything meaningful for the reader, only taking space. Instead, I would provide the means and standard deviations of your distributions for both groups. I would also explicitly mention the test you are doing. So taking your example I would write something along these lines:

Managers are more likely to arrive late than juniors (managers: $10 \pm 5$ minutes late, juniors: $2\pm4$ minutes late, mean$\pm$SD, $N=10$ for both groups, p<.01, Mann-Whitney-Wilcoxon ranksum test)

That's a lot of information to put inside one pair of brackets, so you can split as you like. For example you can report N in the methods section, and make a boxplot figure to illustrate the distributions. Then it would suffice to write:

Managers are more likely to arrive late than juniors, see Figure 1 (p<.01, Mann-Whitney-Wilcoxon ranksum test)

Update

Note that if your data have gross outliers, than means and SDs do not have a lot of meaning and you should rather not report them. Above I assumed that there are no gross outliers in either of the groups. Otherwise situation is more complex and maybe the best way is to provide a boxplot, without giving any numbers in the text at all.

$\endgroup$
  • 2
    $\begingroup$ But the test here is not comparing means. $\endgroup$ – Nick Cox Feb 14 '14 at 21:01
  • $\begingroup$ @NickCox: no, of course not. Do you mean that it would be better to report median$\pm$SD when doing a ranksum test? I think the best would be to provide a boxplot, but let's imagine there is no space for it (or one wants to provide explicit numbers in addition to the figure). $\endgroup$ – amoeba Feb 14 '14 at 21:05
  • $\begingroup$ @NickCox: I added an update to my answer, but would be curious to hear your opinion on the best practice in such case. $\endgroup$ – amoeba Feb 14 '14 at 21:37
  • $\begingroup$ @amoeba I am using Kruskal-Wallis or One Way Anova tests (according to normality) because a statistics lecturer at my University suggested it would be the best choice for such variables. When there are more groups for example: managers, seniors and juniors and the difference in means is significant, are error bar graphs appropriate to see which means differ significantly? I did use these and was able to identify which mean was different from the means of the others in the group. $\endgroup$ – dissertationhelp Feb 15 '14 at 12:06
  • $\begingroup$ @dissertationhelp: the answer to your question is no. You can of course (and should) look at boxplots to get a feeling of how much difference there is, but if you need a proper statistical test, you should conduct separate two-sample tests (Mann-Whitney) and adjust p-values for multiple comparisons. E.g. if you have three groups, then you first run Kruskal-Wallis, and then you can compare three pairs (groups 1-2, 2-3, 1-3) between each other with Mann-Whitney, but you should multiply your p-values by 3. It's called Bonferroni correction for post-hoc test. $\endgroup$ – amoeba Feb 15 '14 at 14:42
0
$\begingroup$

Following above very useful comments I like to add that the median should be reported instead of the mean.

The statement "Managers are more likely to arrive late than juniors (H=14.338, p<.01)" is incomplete. The only thing it says is that there is a difference between the groups. It does not specify where the difference lies or what the exact difference is. For that purpose medians are reported. Following the scenario in the question I would recommend rephrasing; On a 5-point likert scale managers reported to be more often late than juniors (H = xx, p < .01, MdnManagers = x, MdnJuniors x = )

The statement "A statistically significant difference (H=14.338, p<.01) exists between late arrivals at work by managers and juniors." is fine, but could be more informative.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.