Calculate one median for data from five experimental repetitions

Question

I have an experiment that produces a few thousand data points which I then have repeated 5 times. Now I wonder how to calculate one single median to summarize all of them.

1. I could combine the data from the five repetitions and calculate a median for that.
2. I could calculate a median for each experiment and then average them.
3. I could calculate a median for each experiment and then take the median of those.
4. Some other procedure I haven't thought of.

Which procedure make the most sense as a way to combine the data? How would I then go about producing a confidence interval for the median? I assume some sort of bootstrap procedure makes sense, since the distribution of the data is not normal.

Answer 1

A principled way to do the combination would be to assume a parametric probability model which gives a likelihood for each study and then as the studies are presumably independent, they multiply and with an assumption that they have a (certain) parameter(s) in common, that multiplication provides a combined likelihood from which you can extract an estimate and a standard error/confidence limits. Furthermore, assuming each study has a different parameter value still provides a likelihood and it can be used to do an assessment of that assumption of a common parameter via a likelihood ratio test.

Sounds complicated, but if you assumed a Laplace distribution with known scale it would be very, very convenient and direct. More reasonable assumptions and perhaps even that the parameter is not actually common but just drawn from a common distribution can get very complicated. As an aside, Maynard Keynes was one of the earliest to this, after Gauss, Laplace and Bernoulli.

Here is a simple example:

If one had a sample of n patients who could have an event it may be reasonable to assume the events have a constant and common probability of occurring and that they are independent. The probability model for an individual patient’s event then would be p for the event occurring and (1 − p) for the event not occurring, p being between 0 and 1. The likelihood then is p when the event occurs and (1 − p) when it does not. For a sample of patients who either do or do not experience the event, because they were assumed to be independent, the probability model (and hence the likelihood) is just the multiple of the individual models p^x(1−p)^(n−x), where x is the total number of successes and n is the total number of patients. In meta-analysis, the combination of studies by the multiplication of study likelihoods is therefore just a special case. Here, we multiply first by observations within study and then across studies – p1^x1(1−p1)^(n1−x1) * p2^x2(1−p2)^(n2−x2) and equals p^(x1 + x2)(1−p)^(n1 +n2 –x1-x2) if p assumed the same.

But you can fall back on one of Fisher’s old tricks and simply use t.test methods on the median from each study as if it was just a number. A reference on that would be Follman DA, Proschan MA. Valid Inference in Random Effects Meta-Analysis. Biometrics 1999.