Kruskal-Wallis test: assumption testing and interpretation of the results

Question

There is a chapter on the Kruskal-Wallis (KW) test on the website influentianl points, and there are some quotes I'm not sure I understand correctly:

Quote 1:

Some authors state unambiguously that there are no distributional assumptions, others that the homogeneity of variances assumption applies [...]
If you wish to compare medians or means, then the Kruskal-Wallis test also assumes that observations in each group are identically and independently distributed apart from location. If you can accept inference in terms of dominance of one distribution over another, then there are indeed no distributional assumptions.
[link to chapter]

Quote 2:

...heterogeneous variances will make interpretation of the result more complex...
[link to chapter]

My questions:

1. For instance, I analyze dataset chickwts which is included in base R software (below I included a boxplot of the data) and, say, it meets all required assumptions. How (in practical terms from biologist's point of view) interpretation of Kruskal-Wallis test results changes, if I carry out the KW test as a test for medians and if I run it as a test for stochastic dominance? What can I conclude from the data in both cases?
2. From the quote 2 I imply, I should carry out Levene's/Brown-Forsythe test to check for heteroscedasticity. Am I right? If yes, how the result of Levene's test influences the interpretation of Kruskal-Wallis test?
3. Should I carry out other statistical tests (e.g., Kolmogorov-Smirnov test) or make a special type of plots (e.g., QQ plot for each pair of groups) to check if distributions of data in each group have approximately the same shape?

The dataset:

data(chickwts)
boxplot(weight~feed, data = chickwts, las = 3)

Answer 1

The KW test (also the Mann-Whitney U-test) is essentially always a test for stochastic dominance. What that means is it is testing to see if there exists at least one group such that you would typically get a larger (lesser) value from it than the rest if you drew a value at random from each.

People assume this means that one median or mean must be greater than the other, but that isn't necessarily true. If the shapes and the variances of the distributions are identical (i.e., one group's distribution is just shifted up or down relative to the other), then stochastic dominance implies a greater mean and median (and also a greater third quartile, fifth percentile, etc.). However, if the shapes / variances of the distributions differ, then it isn't necessarily the case. For further discussion of these topics and to see an example where the means are switched, see my answer here: Wilcoxon-Mann-Whitney test giving surprising results. For an example where the medians are equal, but there is nonetheless a stochastically dominant group, consider this:

g1 = c(rep(0, 11), 1:10)                # group 1 has 11 0s, & then 1 to 10
g2 <- g3 <- g4<- c(-10:-1, rep(0, 11))  # the other groups have 11 0s, & -1 to -10
d  = stack(list(g1=g1, g2=g2, g3=g3, g4=g4))
aggregate(values~ind, d, median)        # the median of every group is 0
#   ind values
# 1  g1      0
# 2  g2      0
# 3  g3      0
# 4  g4      0

kruskal.test(values~ind, d)  # the KW test is highly significant nonetheless
#   Kruskal-Wallis rank sum test
# 
# data:  values by ind
# Kruskal-Wallis chi-squared = 28.724, df = 3, p-value = 2.559e-06

---

With this understanding in mind, we can answer your specific questions.

1. If the distributions within each group (of chicks) / condition (feed type) have the same shape and variance, a significant KW test implies there is at least one group that is stochastically greater (lesser) than the others, and its mean (and median, and first quartile, and eighty-eighth percentile, etc.) is higher (lower) than the other groups. If the distributions differ in shape and/or variance, a significant KW test implies there is at least one group that is stochastically greater (lesser) than the others, but its mean (and median, and first quartile, and eighty-eighth percentile, etc.) is not necessarily higher (lower) than the other groups.
2. I would not bother running Levene's test before KW.
3. I would not bother running the Kolmogorov-Smirnov test before KW. Examining qq-plots seems reasonable.

Answer 2

This question (Non-normal distribution even with Kruskal-Wallis test) has a nice summary on the difference between inferring stochastic dominance vs. equality of medians.

To answer your specific questions:

1. The K-W test for chickwts gives

   Kruskal-Wallis rank sum test
   
   data:  weight by feed
   Kruskal-Wallis chi-squared = 37.343, df = 5, p-value = 5.113e-07
   

   As a test of medians, we would infer that the median weight after six weeks for at least one group differs from the median weights of the remaining groups. As a test of stochastic dominance, we can only infer that for at least one group, a randomly chosen member of that group is more likely than not to be heavier than a randomly chosen member from another group.

2. If you intend to use KW as a test of medians, then some sort of heteroscedasticity test is warranted. If you find evidence of heteroscedasticity, then you can only infer stochastic dominance from KW.

3. If you want to use KW as test of medians, checking for distributional similarity is a good idea. I'd choose QQ plots over K-S test since the former doesn't require specifying a specific distribution. If the QQ plots suggest differing distributions, then you can only infer stochastic dominance from KW.