Is the sample quantile unbiased for the true quantile? I would like to find a way to show whether the sample quantile is an unbiased estimator of the true quantiles. Let $F$ be strictly increasing with density function $f$. I will define the $p$-th quantile for $0<p<1$ as $Q(p)=F^{-1}(p)$ and the sample quantile as $$\hat{F}_n^{-1}(p)=\inf\{x:\hat{F}_n(x)\geq p\},$$ where $\hat{F}_n(x)$ is the empirical distribution function, given by $$\hat{F}_n(x)=\frac{1}{n}\sum_{i=1}^n I(X_i \leq x).$$ Based on literature I have read, I expect the sample quantile to be biased, but I am having trouble figuring out how to take the expected value of $\hat{F}_n^{-1}(p)$, particularly since it is defined as the infimum of a set. I do know that the expected value of the empirical distribution function is $F(x)$. Any help or references that could guide me would be greatly appreciated!
 A: I do not think Ben's derivation is completely correct. The asymptotic unbiasedness feature of sample quantile is not distribution free. There is an important assumption that the r.v. has to satisfy: there is a unique solution $x$ to the condition $F(x-) \leq p \leq F(x)$. A counter example:
Let $X_1, \dots, X_n \sim$ $X$ i.i.d. where $X$ is Bernoulli taking $-1$ with probability $0.5$ and $1$ with probability $0.5$. Now let $p = 0.5$, then the theoretical quantile (median) $Q(p)$ should be $-1$. Now for the sample quantile
$$
Q_n(p) = \begin{cases}
      -1, & \sum_{i=1}^n\mathbf{1}_{X_i = -1} \geq n/2 \\
      1, & \sum_{i=1}^n\mathbf{1}_{X_i = -1} < n/2.
    \end{cases}
$$
Therefore,
$$
E(Q_n(p)) = -P\left(\frac{\sum_{i=1}^n\mathbf{1}_{X_i = -1}}{n} \geq 1/2\right) + P\left(\frac{\sum_{i=1}^n\mathbf{1}_{X_i = -1}}{n} < 1/2\right).
$$
Now if we take the limit, because of CLT,
$$
\lim_{n\to\infty}P\left(\frac{\sum_{i=1}^n\mathbf{1}_{X_i = -1}}{n} \geq 1/2\right) = \lim_{n\to\infty}P\left(\frac{\sum_{i=1}^n\mathbf{1}_{X_i = -1}}{n} < 1/2\right) = 0.5.
$$
Therefore, $\lim_{n\to\infty}E(Q_n(p)) = 0 \neq -1$.
In fact, we can also check this with mean(2*(replicate(2000, mean(2*rbernoulli(10^6)-1 ==-1)) >= 0.5)-1) which gave an answer of -0.011. The issue is when the solution to $F(x-) \leq p \leq F(x)$ is not unique, i.e., there are segments where the cdf are flat. The sample quantile will jump around and does not settle down.
A: Assuming that $X_1, X_2, X_3 \sim \text{IID } F$ the empirical distribution function has a scaled binomial distribution:
$$\hat{F}_n(x) \sim \frac{1}{n} \cdot \text{Bin}(n, F(x)).$$
For a given probability value $0 < p < 1$ we will denote the sample quantile as:
$$\hat{Q} \equiv \hat{Q}_n(p) \equiv \inf \{ x \in \mathbb{R} | \hat{F}_n(x) \geqslant p \}.$$
Since the empirical distribution function $\hat{F}_n$ is non-decreasing and right-continuous, we have the event equivalence $\inf \{ x \in \mathbb{R} | \hat{F}_n(x) \geqslant p \} \leqslant q$ if and only if $\hat{F}_n(q) \geqslant p$.  Thus, the distribution function for the sample quantile is:
$$\begin{equation} \begin{aligned}
F_{\hat{Q}}(q) = \mathbb{P}(\hat{Q} \leqslant q) 
= \mathbb{P} \bigg( \inf \{ x \in \mathbb{R} | \hat{F}_n(x) \geqslant p \} \leqslant q \bigg) 
= \mathbb{P} \big( \hat{F}_n(q) \geqslant p \big). \\[6pt]
\end{aligned} \end{equation}$$

In order to look at the bias of the sample quantile as an estimator of the true quantile, we need to look at the expected value of the former.  Using a general expectation rule shown here, the exact expected value of this random variable can be written as the integral:
$$\mathbb{E}(\hat{Q}) 
= \int \limits_{-\infty}^\infty \Big[ \mathbb{I}(q \geqslant 0) - F_{\hat{Q}}(q) \Big] dq
= \int \limits_{-\infty}^\infty \Big[ \mathbb{I}(q \geqslant 0) - \mathbb{P} ( \hat{F}_n(q) \geqslant p ) \Big] dq.$$
This integral is complicated, owing to the scaled binomial distribution for $\hat{F}_n$.  However, as $n \rightarrow \infty$ we have $\hat{F}_n(q) \rightarrow F(q)$, and so if $F$ is continuous at $q$ then we also have $Q(\hat{F}_n(q)) \rightarrow q$.  This gives the asymptotic convergence:
$$\mathbb{E}(\hat{Q}) 
\rightarrow \int \limits_{-\infty}^\infty \Big[ \mathbb{I}(q \geqslant 0) - \mathbb{I} ( q \geqslant Q(p) ) \Big] dq
= \int \limits_{0}^{Q(p)} dq = Q(p),$$
so long as $F$ is continuous at $p$.  Thus, you should expect the sample quantiles to be asymptotically unbiased, except at quantiles corresponding to points of discontinuity of the underlying distribution function.  Obviously we may have non-zero bias for finite samples, with the bias depending on the form of the underlying distribution.
