You ask specifically about using the Kruskal-Wallis test. It may be useful for your data, as for my simulated data below. Also, it may be useful to look at some of the 'Related' pages on this site, listed in a margin. [However, see Note 2 at the end, for a brief demonstration
of a possibly appropriate chi-squared test of score counts. Without seeing your data, I cannot say which test I would use for them.]
Roughly speaking, the Kruskal-Wallis test is an extension of the two-sample Wilcoxon (rank sum) test,
to accommodate $g > 2$ groups with non-normal
continuous values from populations of roughly the same
shape.
The 'continuous' assumption anticipates no tied observations between or within the group observations.
Also, for the kind of data you envision, the shapes
of the distributions of the ordinal observations
may be different. This makes it difficult to treat
tests results just as looking at differences in group medians.
Nevertheless, when the K-W test gives significant P-values, it is often possible to interpret the result
as showing that groups are different.
Here is an example with $g = 4$ groups of artificial
data simulated in R.
set.seed(202)
x1 = sample(1:5, 20, rep=T, p=c(1,1,2,2,3))
x2 = sample(1:5, 20, rep=T, p=c(1,2,2,3,3))
x3 = sample(1:5, 20, rep=T, p=c(2,2,3,3,1))
x4 = sample(1:5, 20, rep=T, p=c(3,3,2,2,1))
x = c(x1,x2,x3,x4)
gp = rep(1:4, each=20)
boxplot(x ~ gp, col="skyblue2", pch=19)
kruskal.test(x ~ gp)
Kruskal-Wallis rank sum test
data: x by gp
Kruskal-Wallis chi-squared = 26.816, df = 3,
p-value = 6.433e-06
With appropriate protections (such as Bonferroni adjusted significance levels) against 'false discovery'
it may be appropriate to look ad hoc at some pairs
of differences. For example, even though the shapes of the distributions for Groups 1 and 4 are differ, it is clear that the scores in Group 1 tend to be larger than those in Group 4.
wilcox.test(x1, x4)
Wilcoxon rank sum test
with continuity correction
data: x1 and x4
W = 354, p-value = 1.697e-05
alternative hypothesis:
true location shift is not equal to 0
Warning message:
In wilcox.test.default(x1, x4) :
cannot compute exact p-value with ties
The P-value, even though not exact, is far below the 1%
level. Furthermore, the boxplots above and the empirical CDF (ECDF) plots below, show how the the two groups differ. Scores in Group 1 stochastically dominate (tend to be larger than) those in Group 4.
Specifically, in the plot below, the ECDF for Group 1 (blue) lies to the right of (hence also below) the ECDF for Group 4 (brown).
hdr="ECDF of Gp 1 (blue) Dominates ECDF of Gp 4"
plot(ecdf(x1), col="blue", main = hdr)
lines(ecdf(x4), col="brown", pch = "o")
Notes: (1) Similarly, after the K-W test, one can show a significant difference between Groups 1 and 3 (with a warning message about ties), but not between Groups 2 and 4.
wilcox.test(x1, x3)$p.val
[1] 1.8489e-05
...
wilcox.test(x2, x4)$p.val
[1] 0.1188832
...
(2) Another possible test is a chi-squared test of the null hypothesis (strongly rejected here) that Groups have similarly distributed scores. Because counts are small, it is necessary to simulate the P-value of the chi-squared test (argument sim=T in R).
TBL = rbind(tabulate(x1), tabulate(x2),
tabulate(x3), tabulate(x4))
TBL
[,1] [,2] [,3] [,4] [,5]
[1,] 0 1 2 6 11
[2,] 3 4 1 10 2
[3,] 7 6 2 4 1
[4,] 1 11 4 3 1
chisq.test(TBL, sim=T)
Pearson's Chi-squared test
with simulated p-value
(based on 2000 replicates)
data: TBL
X-squared = 46.069, df = NA,
p-value = 0.0004998
