Testing for difference in distributions of Likert-scale data that differs only in a median-preserving spread Assume that I have a Likert scale with five categories, coded as {1, 2, 3, 4, 5} and that I have two samples, the first of which is generated by drawing from a uniform distribution over these five categories, such that the probabilities are {.2, .2, .2, .2, .2}, and the second of which is a median-preserving spread of the first of the form {.25, .2, .1, .2, .25}.
Which test would show the data from the first and from the second distribution to differ? A Wilcoxon Mann-Whitney test of course does not work, since it only detects differences in medians. I thought a Kruskal-Wallis test would work, since it compares medians and dispersion, but it also does not detect a difference, presumably due to the symmetry of the shift of probability mass. Now I am at a loss for what test to use.
Thank you for your help with this (for me) puzzling question!
 A: Here are two samples of slightly different sizes, simulated to your specification,
with counts put into a matrix.
set.seed(2020)
x1 = sample(1:5, 200, rep=T)
t1 = tabulate(x1)
x2 = sample(1:5, 250, rep=T, p = c(.25, .2, .1, .2, .25))
t2 = tabulate(x2)

MAT = rbind(t1,t2);  MAT
   [,1] [,2] [,3] [,4] [,5]
t1   38   37   46   39   40
t2   55   59   19   46   71

Then a chi-squared test of homogeneity detects that the two samples are not
homogeneous. The null hypothesis of homogeneity is strongly rejected with
P-value far below 5%.
chisq.test(MAT)

        Pearson's Chi-squared test

data:  MAT
X-squared = 23.331, df = 4, p-value = 0.0001087

As you say, a Wilcoxon test does not find a significant difference.
wilcox.test(x1,x2)

        Wilcoxon rank sum test with continuity correction

data:  x1 and x2
W = 24516, p-value = 0.7187
alternative hypothesis: true location shift is not equal to 0

I should mention that a 2-sample Wilcoxon test works best testing for a shift in
center between two samples of approximately the same shape, and your samples
are of distinctly different shapes. (Also, a Kruskal-Wallis test on only two samples
is very similar to a 2-sample Wilcoxon test.)
par(mfrow=c(1,2))
 hist(x1, br=seq(.5,5.5,by=1), col="skyblue2", main="Sample 1")
 hist(x2, br=seq(.5,5.5,by=1), col="skyblue2", main="Sample 2")
par(mfrow=c(1,1))


