# UMVUE of distribution function $F$ when $X_i\sim F$ are i.i.d random variables

Let $(X_1,X_2,\cdots,X_n)$ be a random sample drawn from a population with distribution function $F$. Is the empirical distribution function $F_n$ the UMVUE of $F$? ( $F$ itself is the parameter of interest.)

For fixed $x\in\mathbb R$, the empirical DF is defined as

\begin{align} F_n(x)&=\frac{1}{n}{P(\text{number of observations}\leqslant x)} \\&=\begin{cases}0&,\text{ if }x<X_{(1)}\\\frac{r}{n}&,\text{ if }X_{(r)}\leqslant x<X_{(r+1)}\,,\quad r=1,2,\cdots,n-1\\1&,\text{ if }x\geqslant X_{(n)}\end{cases} \end{align}

We see that $nF_n(x)\sim\text{Bin}(n,F(x))$, whence

$$E(F_n(x))=F(x)$$ and $$\operatorname{Var}(F_n(x))=\frac{F(x)(1-F(x)}{n}\longrightarrow 0\quad\text{ as }n\uparrow \infty$$

Clearly, $\hat{F_n}(x)$ is an unbiased estimator of $F(x)$.

In fact, $\hat{F_n}(x)$ is a consistent estimator of $F(x)$ since for each $x\in\mathbb R$, $$\hat{F_n}(x)\stackrel{P}{\longrightarrow}F(x)\quad\text{ as }n\uparrow \infty$$

Now the question is whether $\hat F_n(x)$ itself can be seen as a complete sufficient statistic for $F(x)$. The sample $(X_1,\cdots,X_n)$ or the order statistics $(X_{(1)},\cdots,X_{(n)})$ are sufficient for $F(x)$. But what is a complete sufficient statistic for $F(x)$?

In any other problem, we can start with a trivial unbiased estimator of $F(x)$, namely $$h(X_1,\cdots,X_n)=\begin{cases}1&,\text{ if }X_1\leqslant x\\0&,\text{ otherwise }\end{cases}$$

And we find $E(h\mid T)$ where $T$ is a complete sufficient statistic for the family of distributions $\{F_{\theta}:\theta\in\Theta\}$ (i.e. $F$ parametrised by some unknown $\theta$ ). But what happens when $F$ itself is the parameter?

One motivation for this question is this old post on Math.SE:

An estimator for the c.d.f $F$ at a point $x_0$?

Maybe this is a well-known result but I could not find a proper reference. If required, it can be assumed that $F$ is continuous and a population pdf $f$ exists.

Edit.

I had not realised that this problem is in fact related to non-parametric estimation. The following is an extract from Theory of Point Estimation by Lehmann-Casella (2nd edition, pg 109).

The exercise 1.6.33 mentioned here asks to show that for i.i.d $X_i$ with unknown density $f$, the order statistics are complete.

• It may simplify things a bit if you view $F(x)$ as nothing more than the success probability of a specific Bernoulli distribution, and so any results that hold here will apply directly to pointwise estimation of cdfs. – dsaxton Jul 30 '18 at 23:01

Sufficiency: The ordered data can be recovered from the ECDF, so the sufficiency of the former implies the sufficiency of the latter.

Completeness: As you have shown, for any fixed value $$x$$ we have $$nF_n(x) \sim \text{Bin}(n,F(x))$$. Hence, for any measureable function $$g$$ and fixed value $$x$$ we have:

\begin{aligned} \mathbb{E}(g(F_n(x))) &= \mathbb{E} \Big( g \Big( \frac{nF_n(x)}{n} \Big) \Big) \\[6pt] &= \sum_{k=0}^n g \Big( \frac{k}{n} \Big) \cdot \text{Bin}(k|n,F(x)) \\[6pt] &= \sum_{k=0}^n g \Big( \frac{k}{n} \Big) {n \choose k} F(x)^k (1-F(x))^{n-k}. \\[6pt] &= \begin{cases} g(0) & \text{for } F(x) = 0, \\[6pt] (1-F(x))^n \sum \limits_{k=0}^n g ( \frac{k}{n} ) {n \choose k} r(x)^k & \text{for } 0

where $$r(x) = F(x)/(1-F(x))$$. The sum in the middle expression is an $$n$$th degree polynomial in $$r$$. Hence, if $$\mathbb{E}(g(F_n(x))) = 0$$ then it must be the case that the coefficients of this polynomial are all zero, which means that $$g(\tfrac{0}{n}) = g(\tfrac{1}{n}) = ... g(\tfrac{n}{n}) = 0$$. Hence, if $$\mathbb{E}(g(F_n(x))) = 0$$ then we have $$g(F_n(x))=0$$ with probability one. This establishes completeness of the statistic $$F_n(x)$$.

UMVUE: We have established that, for all $$x$$, the statistic $$F_n(x)$$ is an unbiased complete sufficient statistic for $$F(x)$$. It follows from the Lehmann-Scheffé theorem that this statistic is the uniform minimum variance unbiased estimator of the underlying distribution.

• Is it also true that the vector of order statistics $(X_{(1)},\cdots,X_{(n)})$ is complete for $F$? – StubbornAtom Aug 3 '18 at 7:20
• For certain sub-classes of distributions they are; see e.g., Bell, Blackwell and Breiman (1960). – Ben Aug 3 '18 at 7:38
• Is the claim that the ECDF is the UMVUE of population DF true only when the order statistics are complete, like when the distribution of the $X_i$'s is absolutely continuous? Does the extract I added in my post hint at this? – StubbornAtom Aug 6 '18 at 7:05
• I am not an expert on this area of statistics, so all I can say here is that the above proof seems to show that the ECDF is UMVUE without any further assumption. Unless there is an error in the above proof that I am not seeing, this appears to be the case. If you're not sure, I'd suggest doing some further research in the literature to see if this is a well-known result. I did the proof from scratch, so I'm not sure if this is replicating a common result. – Ben Aug 6 '18 at 8:06