8
$\begingroup$

Given samples of two distributions I am looking for a test for median difference (I.e. reject null in favor of evidence that medians are different.) I do not want to assume anything about both distributions. Is there any standard test for this situation?

I know Mood's median test, but I believe it assumes that the distributions are shifted. $F_2(t) = F_1(t-a)$ for some $a \in \mathbb{R}$. I back this claim with these sources:

Link1 Link2 Link3

$\endgroup$
  • 2
    $\begingroup$ Notice to possible answers: Wilcoxon rank sum test (Mann-Whitney test) does not test for differences in medians! $\endgroup$ – Firebug Feb 6 '18 at 19:24
  • 1
    $\begingroup$ +1 @Firebug is almost correct: the ran sum test only tests for median difference under stringent additional assumptions (which violates the OP's stated desiderata). $\endgroup$ – Alexis Feb 6 '18 at 19:47
  • $\begingroup$ @Alexis it is a test of medians when the distribution is symmetric, in which case it is also a test of means. $\endgroup$ – AdamO Feb 6 '18 at 19:54
  • $\begingroup$ @AdamO It is a test of medians when both distributions have the same shape (symmetric or not), and when both distributions have the same variance... that is when it is simply a test of location shift. (And yes, then it happens to also be a test for mean difference.) $\endgroup$ – Alexis Feb 6 '18 at 19:57
  • $\begingroup$ @Alexis down the rabbit hole. I just remembered reading that on R docs, so I thought about posting it haha $\endgroup$ – Firebug Feb 6 '18 at 20:06
6
$\begingroup$

You could consider a permutation test.

median.test <- function(x,y, NREPS=1e4) {
  z <- c(x,y)
  i <- rep.int(0:1, c(length(x), length(y)))
  v <- diff(tapply(z,i,median))
  v.rep <- replicate(NREPS, {
    diff(tapply(z,sample(i),median))
  })
  v.rep <- c(v, v.rep)
  pmin(mean(v < v.rep), mean(v>v.rep))*2
}

set.seed(123)
n1 <- 100
n2 <- 200
## the two samples
x <- rnorm(n1, mean=1)
y <- rexp(n2, rate=1)
median.test(x,y)

enter image description here

Gives a 2 sided p-value of 0.1112 which is a testament to how inefficient a median test can be when we don't appeal to any distributional tendency.

If we used MLE, the 95% CI for the median for the normal can just be taken from the mean since the mean is the median in a normal distribution, so that's 1.00 to 1.18. The 95% CI for the median for the exponential can be framed as $\log(2)/\bar{X}$, which by the delta method is 0.63 to 0.80. Therefore the Wald test is statistically significant at the 0.05 level but the median test is not.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ A warning: "The situation is even worse when basing a test on a difference in sample medians, in the sense that regardless of sample sizes, the asymptotic rejection probability of the permutation test will be α under very stringent conditions, which essentially means only in the case where the underlying distributions are the same." from projecteuclid.org/euclid.aos/1366138199. A better solution according to these authors would be to use a studendized version of the test statistic. $\endgroup$ – Julian Karch Jan 28 at 18:56
4
$\begingroup$

Assuming your outcome is ordinal or interval-valued, you can use the nonparametric median test with k=2. Here's a description from Stata's implementation of it:

The median test examines whether it is likely that two or more samples came from populations with the same median. The null hypothesis is that the samples were drawn from populations with the same median. The alternative hypothesis is that at least one sample was drawn from a population with a different median. The test should be used only with ordinal or interval data. Assume that there are score values for k independent samples to be compared. The median test is performed by first computing the median score for all observations combined, regardless of the sample group. Each score is compared with this computed grand median and is classified as being above the grand median, below the grand median, or equal to the grand median. Observations with scores equal to the grand median can be dropped, added to the “above” group, added to the “below” group, or split between the two groups. Once all observations are classified, the data are cast into a 2xk contingency table, and a Pearson’s chi-squared test or Fisher’s exact test is performed.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ I belive this is Mood's median test. Are you sure that it does not assume that distribution are shifted? $\endgroup$ – Manuel Feb 6 '18 at 18:38
  • $\begingroup$ @Manuel I am not familiar with Mood's median test, but it does seem very similar to what I proposed. However, the shift assumption does not appear among the others in the help file I linked, and it is not clear to me where it would be necessary here. It's possible that I am missing something, but perhaps you can add why you believe that it is needed? $\endgroup$ – Dimitriy V. Masterov Feb 6 '18 at 18:57
  • $\begingroup$ I added to the question some links where they say that shifted distribution is necessary. $\endgroup$ – Manuel Feb 6 '18 at 19:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.