I was reading this article ,article link here, about the Central Limit Theorem, CLT, and how it can be used to determine if a cohort of interest is significantly different than the population (I might have phrased this poorly because I don't know the math terms). Using the CLT, I can calculate the cohort of interest's z-score and look at its p-value and either accept or reject the null hypothesis.

Basically, they do lots of random sampling of the population and use the sample means to determine the population mean. Then, they have a cohort which they're interested in and use the formula below to get the cohorts z-score in order to determine if they can reject the null hypothesis.

enter image description here

$M = \text{sample mean}$
$\mu = \text{population mean}$
${\sigma = \text{population standard deviation}}$
$n = \text{sample size}$

In my problem I have a population and I have a smaller cohort of people, which I will call cohort A, that I'm interested in analyzing. The population size is about 200,000 and cohort A is about 5,500. I am trying to test the hypothesis that cohort A is significantly more active than the population. Cohort A does not belong in the population. Also, in cohort A there are extreme outliers which are greatly shifting the mean. I want to use the median instead, to avoid the influence of the few extreme outliers, and read in this post, post link here, that if I apply the same techniques in the CLT and use the median instead, and a large n, I will get a normal distribution just like in the CLT (or at least that was my interpretation given my not so awesome math skills). If this holds true, can I use the formula in the image above and replace the means with medians in order to calculate cohort A's z-score so I can determine if I should accept or reject a null hypothesis that cohort A is significantly more physically active?

  • $\begingroup$ The accepted answer to the linked question deals with two points: the exact distribution of the sample median (using the beta distribution); and the asymptotic distribution for a large sample from a distribution with a density. Note that the variance of the sample median will usually not be the same as the variance of the sample mean $\endgroup$ – Henry Sep 23 at 11:34

Let's assume a distribution of values that are either $0$ or $1$. If you take lots of samples of this distribution the mean of those will approximately be normally distributed.

If you take lots of samples of this distribution the median of those will virtually always be either $0$ or $1$. So the median stays (mostly) dichotomous and thus will never become normally distributed.

So - no, you are not guaranteed to get approximate normal distribution with your median and therefore inference using $z$-scores is not optimal.

However, the solution to your question of testing properties of the "true median" may still be frequent resampling. There is a technique called "Bootstrapping" that will allow you to reason about medians without having to make assumptions of the median's distribution.

As it is a resampling technique and thus computationally intensive, you are not going to do it by hand without software. It is very easy to do bootstraping in R, I think you have to buy special modules to do it in SPSS (but my information may be far outdated) and I have no idea about other software but assume that no professional statistics software theses days could do without bootstrapping functionality.

Let's have an example in R. Say we gathered answers on a 5-point answering scale and it looks like this:

a <- c(1, 1, 1, 1, 1, 1, 1, 2, 3, 3, 4, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5)

These are $21$ observations with a median of $4$. We want to investigate or make inferences on the distribution of plausible medians. We can draw $100,000 = 10^5$ bootstrap samples and compute their medians. It's a one-liner in R:

r <- replicate(1e5, median(sample(a, replace = TRUE)))

The median in this very large number of prepresentative resamples of a is distributed like this:

> table(r)
    1     2     3     4     5 
 5617  7364 28262 45574 13183 

So from $100,000$ resamples we conclude $5.6\%$ have median $1$, $7.3\%$ have median $2$, $28.2\%$ have median $3$ and so on.

We can resample again (takes a second or two) and see, that we get very similar results:

> r <- replicate(1e5, median(sample(a, replace = TRUE)))
> print(table(r))
    1     2     3     4     5 
 5594  7620 28219 45649 12918

So resampling gives us a good understanding of the underlying distribution of the median without any assumptions of normality. In fact the median in this example follows a discrete distribution and thus not a normal distribution. Not even asymptotically.

For easier searching, the bootstrap has its own tag:

| cite | improve this answer | |
  • 1
    $\begingroup$ I understand this could happen if I only had values of 0 and 1. However, I have many values and have extreme outliers that were throwing off my mean pretty badly. That is why I thought using the median would make more sense. Ultimately, what I'm trying to do is determine if my sample group is different than the population using the p-value from the z-score. I didn't get a chance yet to bootstrap the median to see if I get a normal distribution though but if I do have a normal distribution can I use the medians instead to calculate the z-score and then use the corresponding p-value? $\endgroup$ – zipline86 Sep 23 at 10:55
  • 2
    $\begingroup$ By the way, great way to explain how to bootstap using R. Much appreciated and helpful for many to see. $\endgroup$ – zipline86 Sep 23 at 11:00
  • $\begingroup$ Just realized though, when I bootstrap for a problem like this, wouldn't I want to set replacement to False if I a large population? $\endgroup$ – zipline86 Sep 23 at 11:02
  • 2
    $\begingroup$ No. You definitively want to set replace = TRUE because otherwise you'd get the same median over and over again, no matter the sample size. Sorry if that sounds blunt, it's not meant to: The question in itself indicates, that you need to concern yourself more with the bootstrap before considering to actually use it. Concerning p-values: The bootstrap lends itself more to the construction of confidence intervalls then p-values. I you need the p-value to reject some null hypothesis, you can do that with confidence intervalls. $\endgroup$ – Bernhard Sep 23 at 11:41
  • 1
    $\begingroup$ Let me add the following: If outliers are your only problem and you accept the choice of bootstrapping then the median is not your only option: a trimmed mean would be at least a viable option as it feels more like a mean and is usually easily explained to any audience. As for R code you could simply switch median to meanand add the trim = ... argument. Considering the Edit of your question If you have lots of data from the population and little from cohort A you should bootstrap the small sample. Or you bootstrap CIs of both and see whether CIs overlap. $\endgroup$ – Bernhard Sep 24 at 6:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.