21
$\begingroup$

Suppose we have a random variable $X$ supported on $[0,1]$ from which we can draw samples. How can we come up with an unbiased estimate of the median of $X$?

We can, of course, generate some samples and take the sample median, but I understand this will not in general be unbiased.

Note: this question is related, but not identical, to my last question, in which case $X$ could only be sampled approximately.

Improve this question
$\endgroup$

4 Answers 4

Reset to default
16
$\begingroup$

Such an estimator does not exist.

The intuition is that the median can stay fixed while we freely shift probability density around on both sides of it, so that any estimator whose average value is the median for one distribution will have a different average for the altered distribution, making it biased. The following exposition gives a little more rigor to this intuition.

We focus on distributions $F$ having unique medians $m$, so that by definition $F(m) \ge 1/2$ and $F(x) \lt 1/2$ for all $x \lt m$. Fix a sample size $n \ge 1$ and suppose that $t: [0,1]^n \to [0,1]$ estimates $m$. (It will suffice that $t$ only be bounded, but usually one doesn't seriously consider estimators that produce obviously impossible values.) We make no assumptions about $t$; it does not even have to be continuous anywhere.

The meaning of $t$ being unbiased (for this fixed sample size) is that

$$E_F[t(X_1, \ldots, X_n)] = m$$

for any iid sample with $X_i \sim F$. An "unbiased estimator" $t$ is one with this property for all such $F$.

Suppose an unbiased estimator exists. We will derive a contradiction by applying it to a particularly simple set of distributions. Consider distributions $F = F_{x,y,m, \varepsilon}$ having these properties:

  1. $0 \le x \lt y \le 1$;

  2. $0 \lt \varepsilon \lt (y-x)/4$;

  3. $x + \varepsilon \lt m \lt y - \varepsilon$;

  4. $\Pr(X = x) = \Pr(X = y) = (1-\varepsilon)/2$;

  5. $\Pr(m-\varepsilon \le X \le m+\varepsilon) = \varepsilon$; and

  6. $F$ is uniform on $[m-\varepsilon, m+\varepsilon]$.

These distributions place probability $(1-\varepsilon)/2$ at each of $x$ and $y$ and a tiny amount of probability symmetrically placed around $m$ between $x$ and $y$. This makes $m$ the unique median of $F$. (If you are concerned that this is not a continuous distribution, then convolve it with a very narrow Gaussian and truncate the result to $[0,1]$: the argument will not change.)

Now, for any putative median estimator $t$, an easy estimate shows that $E[t(X_1, X_2, \ldots, X_n)]$ is strictly within $\varepsilon$ of the average of the $2^n$ values $t(x_1, x_2, \ldots, x_n)$ where the $x_i$ vary over all possible combinations of $x$ and $y$. However, we can vary $m$ between $x + \varepsilon$ and $y - \varepsilon$, a change of at least $\varepsilon$ (by virtue of conditions 2 and 3). Thus there exists an $m$, and whence a corresponding distribution $F_{x,y,m,\varepsilon}$, for which this expectation does not equal the median, QED.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ (+1) Nice proof. Did you come up with it, or is it something you remembered from the grad school? $\endgroup$
    – StasK
    Sep 12, 2012 at 20:45
  • 7
    $\begingroup$ Here is another proof: Most Bernoulli random variables have median $0$ or $1$. The estimate from $n$ trials depends only on the average values of the estimator on the vertices of $[0,1]^n$ with $k$, and the weights of these average values is a polynomial in $p$ of degree $n$. If this is an unbiased estimator, it must have average value $1$ for any $p \gt 1/2$, and there are more than $n+1$ such values of $p$, so this polynomial must be constant... but it must be $0$ on lower values of $p$, so it can't be unbiased there, too. $\endgroup$
    – Douglas Zare
    Sep 13, 2012 at 3:52
  • 2
    $\begingroup$ @Douglas That's a great proof. I suspect some people might feel a little uneasy about the scope of its applicability, though, because the median for a Bernoulli variable is somewhat special, being coincident with one of its two support points (except when $p=1/2$). Readers might be tempted to declare this as "pathological" and try to bar such monsters by looking only at continuous distributions with everywhere positive densities on their domains. That's why I took care to show that such efforts will fail. $\endgroup$
    – whuber ♦
    Sep 13, 2012 at 16:41
4
$\begingroup$

Finding an unbiased estimator without having a parametric model would be difficult! But you could use bootstrapping, and use that to correct the empirical median to get an approximately unbiased estimator.

Improve this answer
$\endgroup$
6
  • $\begingroup$ If this is impossible, is it possible to prove it? For example, if $X_1, X_2, \ldots, X_n$ are independent samples from $X$ then can one prove that $f(X_1, \ldots, X_n)$ cannot be unbiased for any choice of $f$? $\endgroup$
    – robinson
    Sep 11, 2012 at 22:23
  • 2
    $\begingroup$ I think kjetil is saying that in a nonparametric framework there is no method that will give an unbiased estimate for every possible distribution. But in the parametric framework you probably could. Bootstrapping a biased sample estimate can allow you to estimate the bias and adjust it to get a bootstrap estimate that is nearly unbiased. That was his suggestion for handling the problem in the nonparametric framework. Proving that an unbiased estimate is not possible would also be difficult. $\endgroup$
    – Michael R. Chernick
    Sep 11, 2012 at 23:02
  • 2
    $\begingroup$ If you really want to try to prove that there do not exist an unbiased estimator, there is a book, Ferguson: "Mathematical Statistics - A Decision Theoretic Approach" which do have some examples of that kind of thing! $\endgroup$
    – kjetil b halvorsen ♦
    Sep 12, 2012 at 0:29
  • 3
    $\begingroup$ @Stas As I pointed out, my functions can be made to look very "nice" by mollifying them. They can also be generalized to mollifications of large finite mixtures of atoms. The class of such distributions is dense in all distributions on the unit interval, so I don't think bootstrap regularity would be involved here. $\endgroup$
    – whuber ♦
    Sep 13, 2012 at 16:44
  • 1
    $\begingroup$ @whuber, you are right again. $\endgroup$
    – StasK
    Sep 13, 2012 at 17:03
1
$\begingroup$

I believe quantile regression will give you a consistent estimator of the median. Given the model $Y = \alpha + u$. And you want to estimate $\text{med}(y) = \text{med}(\alpha + u) = \alpha + \text{med}(u)$ since $\alpha$ is a constant. All you need is the $\text{med}(u) = 0$ which should be true so long as you have independent draws. However, as far as unbiasedness, I don't know. Medians are difficult.

Improve this answer
$\endgroup$
1
  • 2
    $\begingroup$ See @whuber 's answer $\endgroup$
    – Peter Flom
    Sep 30, 2012 at 21:36
0
$\begingroup$

The average is not the exclusive statistic that can be used to measure the central tendency of the bias of an estimator, and by restricting yourself to the average the problem ends up being the equivalent of saying that the average does not equal the median.

Take n=1 (just sample populations consisting of a single sample) so that the only value an estimator t of the median m of the population distribution X, t(X_1), can have is X_1, thus E(t(X_1))=E(X_1)=E(X)!=m, therefore it seems there can be no unbiased estimator of m. Yet median(t(X_1))=median(X_1)=median(X)=m... therefore an unbiased estimator does exist: the sample median!

Improve this answer
$\endgroup$
8
  • $\begingroup$ -1 The bias of an estimator is defined to be the difference between its expected value and the parameter it estimates. By changing the definition you do indeed trivialize the problem. $\endgroup$
    – whuber ♦
    Apr 28, 2020 at 12:14
  • $\begingroup$ @whuber yeah I totally abused robinson's unspecified notion of "unbiased", but I had fun doing it. In other words: I disagree. The solution is not so trivial if instead of estimating the median you had to estimate the first quartile, because then the estimate's distance from the median has to be multiplied by n/(n-1) (could be wrong, poor memory). $\endgroup$
    – halsor5
    Apr 28, 2020 at 15:16
  • $\begingroup$ You are welcome to disagree provided you can support your position. What authoritative reference can you point to that employs your definition of bias in estimators? $\endgroup$
    – whuber ♦
    Apr 28, 2020 at 17:56
  • $\begingroup$ +1, just because I hadn't paused to think about the fact that the bias is defined on the average value of the estimator. $\endgroup$
    – ryu576
    Oct 6, 2021 at 23:30
  • $\begingroup$ @whuber there are mean-unbiased estimators and there are median-unbiased estimators and there are many other kinds of unbiased estimators. If the type of bias is not defined it is impossible for there to be a specific meaning "by definition", as you say, because the type of bias itself was not specified to begin with. $\endgroup$
    – halsor5
    Feb 17 at 2:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.