Picking a specific estimated CDF from a set of CDFs provided by an ECDF

Question

Let $F_X$ be a CDF of an unknown random variable $X$. If we have independent samples $x_1, x_2, \ldots, x_n$ of $X$ then we can estimate $F_X$ non-parametrically using an ECDF $\hat{F}_n$. By Central Limit Theorem, for each $i \in \{1,2,\ldots,n\}$

$$\sqrt{n}(\hat{F_n}(x_i) - F_X(x_i)) \rightarrow N(0,F_X(x_i)[1-F_X(x_i)])$$

What I want to know : How to pick one estimated CDF using the distribution above?

Aside from $\hat{F}_n$, I'm also interested in other estimated CDFs based on the distribution per $x$. Independent sampling for each $x$ will most likely violate the non-decreasing requirement of a CDF and exceed the range $[0,1]$.

If we do this parametrically, I just need the likelihood of each parameter given the samples then use a sampling method to pick a specific set parameters to get the CDF. However, I don't have any clue what the distribution is so I'm using a non-parametric approach.

Answer 1

I think you are operating under some misapprehensions here. Firstly, we estimate unknown things from the data we have, not from data we might have gotten instead. Every statistical estimator is a function that maps all possible data vectors we can observe to the space of possible values of the thing we are estimating --- we obtain the estimate by substituting the observed data into this function. So the estimator to use is the empirical CDF of the observed data.

Secondly, you will also find that irrespective of the data, the ECDF is always a valid distribution function (i.e., it is always non-decreasing, right-continuous, etc.). This is guaranteed by the form:

$$\hat{F}_n(x) \equiv \frac{1}{n} \sum_{i=1}^n \mathbb{I}(x_i \leqslant x).$$

It is simple to establish that this function satisfies the requirements of a distribution function for all $\mathbf{x}_n \in \mathbb{R}^n$, so you are incorrect to think that any of these properties will be violated merely because the sample values are IID given the underlying distribution function $F$.

Now, you are correct that the CLT gives the asymptotic normal distribution you have stated. A small variation in this statement allows you to get a pivotal quantity for the estimation problem, which will allow you to form a confidence interval for $F(x)$ using the estimated value $\hat{F}_n(x)$. This is essentially just a confidence interval for a proportion using binomial data, so standard forms apply. (There are a number of different forms for confidence intervals for binomial proportions, but I am a fan of the Wilson score interval.) That is what we use this distributional result for --- it does not imply that we would use a different set of data for the estimate than the data that is actually observed.