# Fitting parametric CDF to ecdf

There is a random variable $X$, but the only data I observed is actually its empirical distribution function (at a suitably fine grid). That is, I only observe $\hat{F}(x)$:=$\#\{x\leq u\}\over N$ for multiple $u$'s, where sample size N is fixed but unknown, and the $u$'s are suitably finely spaced.

I suspect $X$ comes from a parametric distritbution $F_X(x;\alpha,\beta,\mu)$ so I want to estimate the parameters $\alpha,\beta,\mu$. The natural way I can think is to minimize some criterion, say, MSE, between $\hat{F}$ and $F_X$.

I have two questions:

1. Besides MSE, is there any other natural criterion to use? (e.g. some sort of empirical version of Kullback Leibler Divergence?)
2. Is there an entirely different approach to this estimation question? Is there literature of fitting parametric cdf through ecdf that I need to know?

I wouldn't use MSE simply because $\hat{F}$ doesn't have constant variance. If you used a weighted MSE that should do better ... but then $\hat{F}_i$ and $\hat{F}_j$ aren't independent, either.

One possibility if you're trying to make $\hat F$ as close to $F$ in a MSE-like sense as possible is you might try minimizing the Anderson-Darling statistic (since it's like a precision-weighted MSE).

To be honest, I'd be inclined to use likelihood to estimate the parameters, if at all feasible -- but not on the ECDF; you want to deal with the data and construct the likelihood (i.e. look at data values and $f$, not $F$ and $\hat{F}$) if you difference back to what's essentially a histogram$^\dagger$ you should be able to get good ML estimates* from that.

$\hspace{3cm}\to$ 4 5 5 5 6 6 6 7 7 7 7 8 9 9 9 9 9 9 ...

* (if slightly approximate because of the discretization)

$\dagger$ - In case it's not obvious how one obtains the sample size in the case where you only have an ecdf and not $n$, let me be explicit about it. The situation you seem to be dealing with (i.e. as "at a suitably fine grid" suggests) we have a finely discretized continuous distribution, one we might reasonably still treat as if it were continuous if only we had the observations.

Consider that at each jump point, the ecdf must increase by some multiple of $1/n$.

The chance that it will always increase by a multiple of $k/n$ ($2/n$ or $3/n$ say) at every jump point would be extremely small indeed (some simple hand calculations or simulations make the point well enough). So we can - with probability extremely close to 1 - infer the exact $n$ by finding the largest $1/n$ consistent with every change in ecdf.

[On the other hand, with a very small sample at a very coarse grid, so that only a few different values will occur, you could get the increase at each point being $2/n$ or perhaps $3/n$ and so run into problems, particularly if you want standard errors. So if we only had 4 bins and the counts in those bins were 2,6,12,4, we'd get $n$ wrong (it would look like the jumps were multiples of 1/12 rather than 1/24, so we'd think the $n_i$ were 1,3,6,2), and so the standard errors - and in some cases, even the point estimates - could be a fair way off. Fortunately, this isn't the case for this question.]

• The Anderson-Darling stats is a useful suggestion. Thank you. Re differencing back to the histogram and MLE from there, that did come across my mind, but i wasn't sure whether the differencing will do more harm than the MLE will make up for. But since my grid is suitably fine, I hope differencing will be ok? Nov 5, 2014 at 4:33
• To get the ecdf you would have essentially summed a histogram. I'm just suggesting going back one step (reversing that summation back to the discretized data). Nov 5, 2014 at 5:44
• That will do no more harm than the original discretization, since it's just getting the discretized data back. Unless you don't know the original sample size (and you should be able to infer that from the ecdf anyway), this is straightforward calculation. Nov 5, 2014 at 6:23
• Glen, the unknown sample size seems to be a significant complication and perhaps a barrier to applying ML (especially if standard errors for the parameters are sought). What are your thoughts about handling that?
– whuber
Nov 5, 2014 at 15:42
• @whuber The jumps in ecdf must all be multiples of $1/n$. While it's possible to get situations where counts in each bin are all even, say, with many small bins that's extremely unlikely. So with even fairly moderate sample size, nearly always the largest $1/n$ consistent with the ecdf will be the correct one. When $n$ is large, and we treat the sample as continuous, typically point estimates won't usually be sensitive to the value of $n$ (you'll typically get very similar estimates whether you think $n$ is 300 or 76800, say), ... Nov 5, 2014 at 21:57