# When to test For Equality of Medians, and when Stochastic Equality?

In statistics, we are very often interested in investigating whether some score (let's say life satisfaction) tends to be bigger in one population (let's say rich people) compared to another population (let's say poor people). Most often, this research question is formalized by testing the null hypothesis of equal means. However, for a comparison of means to be sensible, the variable of interest must be at least on an interval scale. So, for ordinal data, other formalizations should be used.

Two formalizations are generally used for ordinal data. Equality of medians and stochastic equality, which is defined as $$P(XY)$$, where $$X,Y$$ are the random variables representing the two populations. Many papers argue that stochastic equality is the better formalization. The core argument is that the medians can be equal even if the scores in one population are clearly bigger. Consider as an example the following mixture distributions for $$X$$ and $$Y$$. With probability .5 $$X$$ is just $$1$$, and with probability $$.5$$ it is sampled from $$\text{Uniform}[0,1)$$. Similarly, with probability .5 $$Y$$ is just $$1$$, and with probability $$.5$$ it is sampled from $$\text{Uniform}(1,2]$$. Thus, $$\text{Median}(X)=1=\text{Median}(Y)$$, while no realization of $$Y$$ is smaller than any realization of $$X$$ and $$50\%$$ of the realizations of $$Y$$ are bigger than all realizations of $$X$$.

Thus, should we stop testing for equality of medians, or are there research questions for which the equality of medians is the appropriate question? If yes, what are those?

• I like this because it says that asking the right questions is very important. I would look at geometric means instead of medians or means. I might also consider harmonic means when thinking about trajectories over time, especially if trying to select a treatment to engage inequality. Commented Jul 15, 2020 at 14:07
• I would say that stochastic equality is often also a better alternative to tests of means. Many, many papers show that two groups A and B differ on the mean of some variable, but for any such given difference in the mean, people will vastly overestimate how likely it is that a randomly drawn instance of A is larger than one from B. Commented Jul 15, 2020 at 14:12
• The question just is whether this is answerable here at CV. Yes, we should answer the right questions, but which question is the right one will depend on your application. I imagine this might be closed as opinion-based. Commented Jul 15, 2020 at 14:13
• @StephanKolassa: I completely agree with you. Stochastic equality is often also a better alternative to tests of means. This is nicely argued in the literature, and guidelines for choosing between the two are presented, whereas I found nothing for the comparison of equality of medians and stochastic equality. Commented Jul 15, 2020 at 17:18
• I also completely agree that essentially the application should dictate the question you ask. In that sense, my question is, are there applications for which the equality of medians is a sensible question, and if yes can we provide guidelines for practitioners of how to choose between stochastic equality and equality of medians. Commented Jul 15, 2020 at 17:18

This depends, in part, on the number of ordinal categories.

If the number of categories is small, then comparing the medians may be uninformative. Suppose the categories are Like/Like Somewhat/Neutral/Dislike Somewhat/Dislike, the poor group’s answers are distributed 20%-20%-20%-20%-20%, and the rich group’s answers are distributed 10%-20%-25%-25%-20%. Then both groups have a median response of Neutral, but the comparison of random pairs shows that a rich person will probably be more negative than the poor person. Depending on the size of the groups, that might be significant.

If the number of categories is large, then comparing the medians may be more useful. Suppose twelve people from groups A and B all start a task at the same time, and the only data available is that they finish in the order ABABBBAAABAB. Then a random pair from the two groups is equally likely to have the A or B person finish first, but the median person in group B finished earlier than the median person in group A. Again, with enough people in the groups, the difference in medians might be significant.

More generally, if you don’t care about small perturbations in the tails of the two grops, or if you think that there’s more noise or measurement error in those tails, then the medians are more robust and it would be appropriate to test the significance of their difference.