Kruskal-Wallis vs Jonckheere-Terpstra Test If we're doing a Jonckheere-Terpstra Test and a Kruskal-Wallis test at the same significance level, which one is more likely to reject the null hypothesis when it is actually true?
From my understanding, the Jonckheere-Terpstra test tests the ordering of medians versus the Kruskal-Wallis test, that tests the equality of medians. So the former can give a significant result if the medians are actually equal, which Kruskal-Wallis will not. Hence the former is more likely to reject the null than the latter. Does this make sense? An explanation will be helpful.
 A: You are correct that the Kruskal-Wallis test ignores ordering—making it akin to a one-way ANOVA performed on ranks. However, you are a bit off with the null hypothesis of the Kruskal-Wallis which takes the form:
$$H_{0}: P\left(X_{i} > X_{j}\right) = 0.5\\H_{A}: P\left(X_{i} > X_{j}\right) \ne 0.5$$
for all groups $i\ne j$. In plain language the null is that the probability of a randomly selected observation in one group being greater than a randomly selected observation in another group is one half.
The Kruskal-Wallis test only becomes an omnibus test for difference of unordered medians with two additional assumptions:


*

*The distributions of each group all have the same shape—whatever that shape actually is.

*The distributions of each group all have the same variance.


These two strict assumptions make Kruskal-Wallis a test for location shift (more general than a test for mean difference, or a test for median difference). 
Given that the Kruskal-Wallis test ignores ordering, ceteris paribus the Jonckheere-Terpstra test will be more powerful. Indeed, Jonckheere noted that the test in balanced data was more powerful than a one-way ANOVA, which also ignores ordering.
You write "So the former can give a significant result if the medians are actually equal, which Kruskal-Wallis will not," but this is untrue: the Kruskal-Wallis test certainly can reject the null hypothesis even when it is true. In fact, when you choose $\alpha$ as a rejection criterion for this test you are selecting your preference for making exactly this kind of Type I error. Jonckheere gives a caution about using the Jonckheere-Terpstra test when the sample size in one group is large, but very small in other groups, as this will make the limiting distribution of the approximate test statistic non-normal, making inferences unreliable.
