Demonstration of sample quantile bias While doing some simulations, I realised that the sample quantile is a biased estimator of the true quantile. And, according to my simulations, a potentially very biased one.
I was surprised with that result since the empirical CDF isn't biased, but after some internet research, I discovered it was true.
I tried to figure out where that bias comes from, but working with sample quantiles is quite difficult. Does anyone have a demonstration of that bias (and, ideally, a quantification)?
 A: Bias in estimating $p$-quantiles is investigated in a distribution-free way in
http://www.sciencedirect.com/science/article/pii/S016771520000242X
(a pdf can be found on the same page). The authors focus on the quantile estimator based on ECDF inversion. No assumptions on the underlying distribution is made (except finite second moment), thus also discrete distributions are included.
Some highlights:


*

*Bias is proportional to the standard deviation $\sigma$ of the underlying distribution

*Bias is smaller in central quantiles than in extreme ones. This stems from the fact that among all distributions with standard deviation $\sigma < \infty$, bias oscillates in an interval of length $\frac{\sigma}{\sqrt{p (1-p)}}$. Strikingly, this does not depend on the sample size $n$.

*For $np>3$, among all standardized distributions (mean 0, standard deviation 1), the worst bias is associated with the distribution having an atom of probability $p$ at $-\sqrt{(1-p)/p}$ and an atom of probability $1-p$ at $\sqrt{p/(1-p)}$.
A: Just to add to this old post, The ECDF is only unbiased at high samples sizes. At low values of N it is biased. Take the trivial case of N=1 and the ECDF takes a value of 1 at and above the sample value. Ask yourself what is the value of the underlying distribution that gives a probability of 1?
The bias actually exceeds sqrt(2*pi)/(2N)*SD or 1.25/N * SD so for an N of 5 this is a 0.25 SD bias.
Instead of an ECDF based on k/N, try (k-0.5)/N to get an unbiased ECDF.
That might give you unbiased sample quantiles.
It also ensures that ECDF(x)=1-ECDF(-x) which is enjoyed by all other cumulative distributions.
In my very humble opinion the ECDF as defined and used is a huge misnomer.
It biases Kolmogorov Smirnov, Lilliefors and other standard tests at low N.
Check out Gilchrist "Statistical modelling with Quantile Functions"
A: There exists a unique true sample quantile definition (which is not the one usually presented). See: http://dx.doi.org/10.1155/2014/326579
