I would like to find a way to show whether the sample quantile is an unbiased estimator of the true quantiles. Let $F$ be strictly increasing with density function $f$. I will define the $p$-th quantile for $0<p<1$ as $Q(p)=F^{-1}(p)$ and the sample quantile as $$\hat{F}_n^{-1}(p)=\inf\{x:\hat{F}_n(x)\geq p\},$$ where $\hat{F}_n(x)$ is the empirical distribution function, given by $$\hat{F}_n(x)=\frac{1}{n}\sum_{i=1}^n I(X_i \leq x).$$ Based on literature I have read, I expect the sample quantile to be biased, but I am having trouble figuring out how to take the expected value of $\hat{F}_n^{-1}(p)$, particularly since it is defined as the infimum of a set. I do know that the expected value of the empirical distribution function is $F(x)$. Any help or references that could guide me would be greatly appreciated!

  • $\begingroup$ Cross-posted at math.stackexchange.com/q/3378799/321264. $\endgroup$ – StubbornAtom Oct 3 '19 at 4:39
  • $\begingroup$ Asymptotically unbiased for $p$th sample quantile (other than max and min), of a continuous distributions with positive density at $p$th population quantile. Asymptotic dist'n of the sample quantile is normally distributed with pop quantile as mean. (Sort of a CLT for 'central' quantiles.) $\endgroup$ – BruceET Oct 5 '19 at 3:54

Assuming that $X_1, X_2, X_3 \sim \text{IID } F$ the empirical distribution function has a scaled binomial distribution:

$$\hat{F}_n(x) \sim \frac{1}{n} \cdot \text{Bin}(n, F(x)).$$

For a given probability value $0 < p < 1$ we will denote the sample quantile as:

$$\hat{Q} \equiv \hat{Q}_n(p) \equiv \inf \{ x \in \mathbb{R} | \hat{F}_n(x) \geqslant p \}.$$

Since the empirical distribution function $\hat{F}_n$ is non-decreasing and right-continuous, we have the event equivalence $\inf \{ x \in \mathbb{R} | \hat{F}_n(x) \geqslant p \} \leqslant q$ if and only if $\hat{F}_n(q) \geqslant p$. Thus, the distribution function for the sample quantile is:

$$\begin{equation} \begin{aligned} F_{\hat{Q}}(q) = \mathbb{P}(\hat{Q} \leqslant q) = \mathbb{P} \bigg( \inf \{ x \in \mathbb{R} | \hat{F}_n(x) \geqslant p \} \leqslant q \bigg) = \mathbb{P} \big( \hat{F}_n(q) \geqslant p \big). \\[6pt] \end{aligned} \end{equation}$$

In order to look at the bias of the sample quantile as an estimator of the true quantile, we need to look at the expected value of the former. Using a general expectation rule shown here, the exact expected value of this random variable can be written as the integral:

$$\mathbb{E}(\hat{Q}) = \int \limits_{-\infty}^\infty \Big[ \mathbb{I}(q \geqslant 0) - F_{\hat{Q}}(q) \Big] dq = \int \limits_{-\infty}^\infty \Big[ \mathbb{I}(q \geqslant 0) - \mathbb{P} ( \hat{F}_n(q) \geqslant p ) \Big] dq.$$

This integral is complicated, owing to the scaled binomial distribution for $\hat{F}_n$. However, as $n \rightarrow \infty$ we have $\hat{F}_n(q) \rightarrow F(q)$, and so if $F$ is continuous at $q$ then we also have $Q(\hat{F}_n(q)) \rightarrow q$. This gives the asymptotic convergence:

$$\mathbb{E}(\hat{Q}) \rightarrow \int \limits_{-\infty}^\infty \Big[ \mathbb{I}(q \geqslant 0) - \mathbb{I} ( q \geqslant Q(p) ) \Big] dq = \int \limits_{0}^{Q(p)} dq = Q(p),$$

so long as $F$ is continuous at $p$. Thus, you should expect the sample quantiles to be asymptotically unbiased, except at quantiles corresponding to points of discontinuity of the underlying distribution function. Obviously we may have non-zero bias for finite samples, with the bias depending on the form of the underlying distribution.

  • 2
    $\begingroup$ (+1) Right about bias for small samples. Simulation in R for $10^6$ samples of size $n=5$ from $\mathsf{Exp}(1):$ h = replicate(10^6, median(rexp(5, 1))); mean(h) returns 0.783058, but population median is 0.6931472. $\endgroup$ – BruceET Oct 5 '19 at 4:03
  • 1
    $\begingroup$ @BruceET: an exception being the median of a symmetric distribution: for instance, mean(replicate(1e6,median(rnorm(5)))) returns -0.001093016 when mean(replicate(1e6,mean(rnorm(5)))) returns -0.0001424392. $\endgroup$ – Xi'an Apr 29 '20 at 14:01

I do not think Ben's derivation is completely correct. The asymptotic unbiasedness feature of sample quantile is not distribution free. There is an important assumption that the r.v. has to satisfy: there is a unique solution $x$ to the condition $F(x-) \leq p \leq F(x)$. A counter example:

Let $X_1, \dots, X_n \sim$ $X$ i.i.d. where $X$ is Bernoulli taking $-1$ with probability $0.5$ and $1$ with probability $0.5$. Now let $p = 0.5$, then the theoretical quantile (median) $Q(p)$ should be $-1$. Now for the sample quantile $$ Q_n(p) = \begin{cases} -1, & \sum_{i=1}^n\mathbf{1}_{X_i = -1} \geq n/2 \\ 1, & \sum_{i=1}^n\mathbf{1}_{X_i = -1} < n/2. \end{cases} $$ Therefore, $$ E(Q_n(p)) = -P\left(\frac{\sum_{i=1}^n\mathbf{1}_{X_i = -1}}{n} \geq 1/2\right) + P\left(\frac{\sum_{i=1}^n\mathbf{1}_{X_i = -1}}{n} < 1/2\right). $$ Now if we take the limit, because of CLT, $$ \lim_{n\to\infty}P\left(\frac{\sum_{i=1}^n\mathbf{1}_{X_i = -1}}{n} \geq 1/2\right) = \lim_{n\to\infty}P\left(\frac{\sum_{i=1}^n\mathbf{1}_{X_i = -1}}{n} < 1/2\right) = 0.5. $$ Therefore, $\lim_{n\to\infty}E(Q_n(p)) = 0 \neq -1$.

In fact, we can also check this with mean(2*(replicate(2000, mean(2*rbernoulli(10^6)-1 ==-1)) >= 0.5)-1) which gave an answer of -0.011. The issue is when the solution to $F(x-) \leq p \leq F(x)$ is not unique, i.e., there are segments where the cdf are flat. The sample quantile will jump around and does not settle down.

  • $\begingroup$ Well-spotted (+1) --- I have amended my answer to add the required continuity condition. $\endgroup$ – Ben Oct 5 '20 at 20:21
  • $\begingroup$ Actually I shouldn't say continuous (I corrected it). It should be there is only a unique solution $x$ to the condition $F(x-) \leq p \leq F(x)$. Because you can also have $X\sim unif(0,1)\cup(2,3)$ and pick $p = 0.5$. This is similar to the counterexample I gave. The true median is 1 but the asymptotic expectiation of sample quantile go to $1.5$. But $X$ here still has continuous cdf everywhere. $\endgroup$ – NamelessGods Oct 5 '20 at 21:57
  • $\begingroup$ @NamelessGods In your comment example, I would say the median is any value from $1$ through to $2$, including $1.5$, and in your main example the median is any value from $-1$ through to $+1$, including $0$ $\endgroup$ – Henry Oct 6 '20 at 8:13
  • $\begingroup$ I don't think it is any value in (-1,1) as proven in my main post.The sample median in my main post is either -1 or 1. But the asymptotic median is certainly 0. $\endgroup$ – NamelessGods Oct 6 '20 at 20:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.