An unbiased estimate of the median

Suppose we have a random variable $X$ supported on $[0,1]$ from which we can draw samples. How can we come up with an unbiased estimate of the median of $X$?

We can, of course, generate some samples and take the sample median, but I understand this will not in general be unbiased.

Note: this question is related, but not identical, to my last question, in which case $X$ could only be sampled approximately.

-

Such an estimator does not exist.

The intuition is that the median can stay fixed while we freely shift probability density around on both sides of it, so that any estimator whose average value is the median for one distribution will have a different average for the altered distribution, making it biased. The following exposition gives a little more rigor to this intuition.

We focus on distributions $F$ having unique medians $m$, so that by definition $F(m) \ge 1/2$ and $F(x) \lt 1/2$ for all $x \lt m$. Fix a sample size $n \ge 1$ and suppose that $t: [0,1]^n \to [0,1]$ estimates $m$. (It will suffice that $t$ only be bounded, but usually one doesn't seriously consider estimators that produce obviously impossible values.) We make no assumptions about $t$; it does not even have to be continuous anywhere.

The meaning of $t$ being unbiased (for this fixed sample size) is that

$$E_F[t(X_1, \ldots, X_n)] = m$$

for any iid sample with $X_i \sim F$. An "unbiased estimator" $t$ is one with this property for all such $F$.

Suppose an unbiased estimator exists. We will derive a contradiction by applying it to a particularly simple set of distributions. Consider distributions $F = F_{x,y,m, \varepsilon}$ having these properties:

1. $0 \le x \lt y \le 1$;

2. $0 \lt \varepsilon \lt (y-x)/4$;

3. $x + \varepsilon \lt m \lt y - \varepsilon$;

4. $\Pr(X = x) = \Pr(X = y) = (1-\varepsilon)/2$;

5. $\Pr(m-\varepsilon \le X \le m+\varepsilon) = \varepsilon$; and

6. $F$ is uniform on $[m-\varepsilon, m+\varepsilon]$.

These distributions place probability $(1-\varepsilon)/2$ at each of $x$ and $y$ and a tiny amount of probability symmetrically placed around $m$ between $x$ and $y$. This makes $m$ the unique median of $F$. (If you are concerned that this is not a continuous distribution, then convolve it with a very narrow Gaussian and truncate the result to $[0,1]$: the argument will not change.)

Now, for any putative median estimator $t$, an easy estimate shows that $E[t(X_1, X_2, \ldots, X_n)]$ is strictly within $\varepsilon$ of the average of the $2^n$ values $t(x_1, x_2, \ldots, x_n)$ where the $x_i$ vary over all possible combinations of $x$ and $y$. However, we can vary $m$ between $x + \varepsilon$ and $y - \varepsilon$, a change of at least $\varepsilon$ (by virtue of conditions 2 and 3). Thus there exists an $m$, and whence a corresponding distribution $F_{x,y,m,\varepsilon}$, for which this expectation does not equal the median, QED.

-
(+1) Nice proof. Did you come up with it, or is it something you remembered from the grad school? –  StasK Sep 12 '12 at 20:45
Here is another proof: Most Bernoulli random variables have median $0$ or $1$. The estimate from $n$ trials depends only on the average values of the estimator on the vertices of $[0,1]^n$ with $k$, and the weights of these average values is a polynomial in $p$ of degree $n$. If this is an unbiased estimator, it must have average value $1$ for any $p \gt 1/2$, and there are more than $n+1$ such values of $p$, so this polynomial must be constant... but it must be $0$ on lower values of $p$, so it can't be unbiased there, too. –  Douglas Zare Sep 13 '12 at 3:52
@Douglas That's a great proof. I suspect some people might feel a little uneasy about the scope of its applicability, though, because the median for a Bernoulli variable is somewhat special, being coincident with one of its two support points (except when $p=1/2$). Readers might be tempted to declare this as "pathological" and try to bar such monsters by looking only at continuous distributions with everywhere positive densities on their domains. That's why I took care to show that such efforts will fail. –  whuber Sep 13 '12 at 16:41

Finding an unbiased estimator without having a parametric model would be difficult! But you could use bootstrapping, and use that to correct the empirical median to get an approximately unbiased estimator.

-
If this is impossible, is it possible to prove it? For example, if $X_1, X_2, \ldots, X_n$ are independent samples from $X$ then can one prove that $f(X_1, \ldots, X_n)$ cannot be unbiased for any choice of $f$? –  robinson Sep 11 '12 at 22:23
I think kjetil is saying that in a nonparametric framework there is no method that will give an unbiased estimate for every possible distribution. But in the parametric framework you probably could. Bootstrapping a biased sample estimate can allow you to estimate the bias and adjust it to get a bootstrap estimate that is nearly unbiased. That was his suggestion for handling the problem in the nonparametric framework. Proving that an unbiased estimate is not possible would also be difficult. –  Michael Chernick Sep 11 '12 at 23:02
If you really want to try to prove that there do not exist an unbiased estimator, there is a book, Ferguson: "Mathematical Statistics - A Decision Theoretic Approach" which do have some examples of that kind of thing! –  kjetil b halvorsen Sep 12 '12 at 0:29
I imagine that the regularity conditions for the bootstrap will be violated with the distribution functions that whuber considers in his answer. Michael, can you comment? –  StasK Sep 12 '12 at 20:44
@Stas As I pointed out, my functions can be made to look very "nice" by mollifying them. They can also be generalized to mollifications of large finite mixtures of atoms. The class of such distributions is dense in all distributions on the unit interval, so I don't think bootstrap regularity would be involved here. –  whuber Sep 13 '12 at 16:44
show 1 more comment

I believe quantile regression will give you a consistent estimator of the median. Given the model $Y = \alpha + u$. And you want to estimate $\text{med}(y) = \text{med}(\alpha + u) = \alpha + \text{med}(u)$ since $\alpha$ is a constant. All you need is the $\text{med}(u) = 0$ which should be true so long as you have independent draws. However, as far as unbiasedness, I don't know. Medians are difficult.

-
See @whuber 's answer –  Peter Flom Sep 30 '12 at 21:36