I am given a sample of data for which I need to test the hypothesis:

$H_0: m=m_0$ vs. $H_1: m \neq m_0$

where $m$ is the population median and $m_0$ is the hypothesized median.

The question asks to construct a randomisation test to test this. What I don't understand is that in class the randomisation tests we've done has involved permuting data and constructing a new test statistic for each permutation, and comparing the distribution of those test statistics to our data.

However, if I permute this data it will just have the same median obviously, so how should I go about this?

I thought that I should take samples with replacement from our data, calculate the median for each sample, and then compare the median of the data with the distribution of generated medians. However this method of using bootstrap doesn't appear to come under "randomisation techniques" in our notes, which the question specifically asks us to use.

  • $\begingroup$ This does seem like a strange question. The idea behind the randomization test is to empirically estimate the null distribution of a test statistic, but in this situation there are tests with a fully known null distribution, e.g., the sign test. Estimating it using randomization would be a waste of time. $\endgroup$
    – dsaxton
    Sep 24, 2016 at 0:28
  • $\begingroup$ The first part of the question was the sign test, and now we need to it with a randomisation test to compare the results. I'm confused though because all randomisation tests we did involved covariates you could permute. I dont understand how permutation could be used to get different test statistics when we only have data without any covariates. $\endgroup$
    – Patty
    Sep 24, 2016 at 0:44

1 Answer 1


This is perfectly simple. You just need a suitable test statistic! One thing you need is that the test statistic will behave differently when the alternative is true that when the null is true.

There's a variety of ways to approach it, but you may find with a little thought that they tell you the same thing.

a) One approach is to take the difference between the count above $m_0$ and the count $\leq m_0$. This will tend to be close to $0$ when $H_0$ is true (and the distribution is continuous) and tend not to be close to $0$ when $H_0$ is false. You might then choose to have your rejection region be the cases when the absolute value of the test statistic is largest.

b) You could just count how many sample values are $\leq m_0$ (call this $X$ say). If the null is true and the distribution is continuous this should be close to $n/2$. Otherwise it might tend to be larger or smaller. You need some way to chose which permutations in those two tails are at least as extreme as each other; if you use distance from $n/2$ that would be the same as using $T=|X-n/2|$ as a test statistic.

[There are several other ways you could do it]

So what do you randomize here? If the null is true, each observation is equally likely to be above $m_0$ (for which its contribution to the difference in counts will be +1) as below (for which its contribution to the difference in counts will be -1). Which is to say you randomly assign each observation to be either in the group "above the median" or "below the median" ("toss a coin", in effect). There are a bunch of ways to do this as well.

If instead of the difference in counts above and below you used the count below $m_0$ then the contributions to the count would be 0 and 1 with equal probability, so for each observation you'd generate a random set of 0's and 1's (and this is where the equivalence to the sign test should now be completely plain).

Hopefully that's enough for you to begin (as well as to see why it should give you the same information as a sign test does).

  • $\begingroup$ So just to make sure I understand this correctly. One way of performing a randomisation test would be to generate a bunch of Binomial(n,0.5) random variables and compare the actual number less than the median to this generated distribution? $\endgroup$
    – Patty
    Sep 24, 2016 at 9:23
  • 1
    $\begingroup$ That's what it effectively boils down to, but of course in practice if you realize this, you also realize that a proper permutation test using the full permutation distribution is available just by using binomial tables. More generally this recognition won't occur (it won't be something as simple as a binomial) but the procedure I outlined works (i.e. find a suitable test statistic that has a different behavior under the alternative than it does under the null and figure out a suitable permutation scheme that should leave the distribution of the test statistic unchanged when the null is true) $\endgroup$
    – Glen_b
    Sep 25, 2016 at 10:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.