3
$\begingroup$

Let $(X_1,X_2,\cdots,X_n)$ be a random sample drawn from a population with distribution function $F$. Is the empirical distribution function $F_n$ the UMVUE of $F$? ( $F$ itself is the parameter of interest.)

For fixed $x\in\mathbb R$, the empirical DF is defined as

\begin{align} F_n(x)&=\frac{1}{n}{P(\text{number of observations}\leqslant x)} \\&=\begin{cases}0&,\text{ if }x<X_{(1)}\\\frac{r}{n}&,\text{ if }X_{(r)}\leqslant x<X_{(r+1)}\,,\quad r=1,2,\cdots,n-1\\1&,\text{ if }x\geqslant X_{(n)}\end{cases} \end{align}

We see that $nF_n(x)\sim\text{Bin}(n,F(x))$, whence

$$E(F_n(x))=F(x)$$ and $$\operatorname{Var}(F_n(x))=\frac{F(x)(1-F(x)}{n}\longrightarrow 0\quad\text{ as }n\uparrow \infty$$

Clearly, $\hat{F_n}(x)$ is an unbiased estimator of $F(x)$.

In fact, $\hat{F_n}(x)$ is a consistent estimator of $F(x)$ since for each $x\in\mathbb R$, $$\hat{F_n}(x)\stackrel{P}{\longrightarrow}F(x)\quad\text{ as }n\uparrow \infty$$

Now the question is whether $\hat F_n(x)$ itself can be seen as a complete sufficient statistic for $F(x)$. The sample $(X_1,\cdots,X_n)$ or the order statistics $(X_{(1)},\cdots,X_{(n)})$ are sufficient for $F(x)$. But what is a complete sufficient statistic for $F(x)$?

In any other problem, we can start with a trivial unbiased estimator of $F(x)$, namely $$h(X_1,\cdots,X_n)=\begin{cases}1&,\text{ if }X_1\leqslant x\\0&,\text{ otherwise }\end{cases}$$

And we find $E(h\mid T)$ where $T$ is a complete sufficient statistic for the family of distributions $\{F_{\theta}:\theta\in\Theta\}$ (i.e. $F$ parametrised by some unknown $\theta$ ). But what happens when $F$ itself is the parameter?

One motivation for this question is this old post on Math.SE:

An estimator for the c.d.f $F$ at a point $x_0$?

Maybe this is a well-known result but I could not find a proper reference. If required, it can be assumed that $F$ is continuous and a population pdf $f$ exists.

Edit.

I had not realised that this problem is in fact related to non-parametric estimation. The following is an extract from Theory of Point Estimation by Lehmann-Casella (2nd edition, pg 109).

The exercise 1.6.33 mentioned here asks to show that for i.i.d $X_i$ with unknown density $f$, the order statistics are complete.

enter image description here

$\endgroup$
  • $\begingroup$ It may simplify things a bit if you view $F(x)$ as nothing more than the success probability of a specific Bernoulli distribution, and so any results that hold here will apply directly to pointwise estimation of cdfs. $\endgroup$ – dsaxton Jul 30 '18 at 23:01
2
$\begingroup$

Sufficiency: The ordered data can be recovered from the ECDF, so the sufficiency of the former implies the sufficiency of the latter.

Completeness: As you have shown, for any fixed value $x$ we have $nF_n(x) \sim \text{Bin}(n,F(x))$. Hence, for any measureable function $g$ and fixed value $x$ we have:

$$\begin{equation} \begin{aligned} \mathbb{E}(g(F_n(x))) &= \mathbb{E} \Big( g \Big( \frac{nF_n(x)}{n} \Big) \Big) \\[6pt] &= \sum_{k=0}^n g \Big( \frac{k}{n} \Big) \cdot \text{Bin}(k|n,F(x)) \\[6pt] &= \sum_{k=0}^n g \Big( \frac{k}{n} \Big) {n \choose k} F(x)^k (1-F(x))^{n-k}. \\[6pt] &= \begin{cases} g(0) & \text{for } F(x) = 0, \\[6pt] (1-F(x))^n \sum \limits_{k=0}^n g ( \frac{k}{n} ) {n \choose k} r(x)^k & \text{for } 0<F(x)<1, \\[6pt] g(1) & \text{for } F(x) = 1, \\[6pt] \end{cases} \end{aligned} \end{equation}$$

where $r(x) = F(x)/(1-F(x))$. The sum in the middle expression is an $n$th degree polynomial in $r$. Hence, if $\mathbb{E}(g(F_n(x))) = 0$ then it must be the case that the coefficients of this polynomial are all zero, which means that $g(\tfrac{0}{n}) = g(\tfrac{1}{n}) = ... g(\tfrac{n}{n}) = 0$. Hence, if $\mathbb{E}(g(F_n(x))) = 0$ then we have $g(F_n(x))=0$ with probability one. This establishes completeness of the statistic $F_n(x)$.

UMVUE: We have established that, for all $x$, the statistic $F_n(x)$ is an unbiased complete sufficient statistic for $F(x)$. It follows from the Lehmann-Scheffé theorem that this statistic is the uniform minimum variance unbiased estimator of the underlying distribution.

$\endgroup$
  • $\begingroup$ Is it also true that the vector of order statistics $(X_{(1)},\cdots,X_{(n)})$ is complete for $F$? $\endgroup$ – StubbornAtom Aug 3 '18 at 7:20
  • $\begingroup$ For certain sub-classes of distributions they are; see e.g., Bell, Blackwell and Breiman (1960). $\endgroup$ – Ben Aug 3 '18 at 7:38
  • $\begingroup$ Is the claim that the ECDF is the UMVUE of population DF true only when the order statistics are complete, like when the distribution of the $X_i$'s is absolutely continuous? Does the extract I added in my post hint at this? $\endgroup$ – StubbornAtom Aug 6 '18 at 7:05
  • $\begingroup$ I am not an expert on this area of statistics, so all I can say here is that the above proof seems to show that the ECDF is UMVUE without any further assumption. Unless there is an error in the above proof that I am not seeing, this appears to be the case. If you're not sure, I'd suggest doing some further research in the literature to see if this is a well-known result. I did the proof from scratch, so I'm not sure if this is replicating a common result. $\endgroup$ – Ben Aug 6 '18 at 8:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.