7
$\begingroup$

There is a random variable $X$, but the only data I observed is actually its empirical distribution function (at a suitably fine grid). That is, I only observe $\hat{F}(x)$:=$\#\{x\leq u\}\over N $ for multiple $u$'s, where sample size N is fixed but unknown, and the $u$'s are suitably finely spaced.

I suspect $X$ comes from a parametric distritbution $F_X(x;\alpha,\beta,\mu)$ so I want to estimate the parameters $\alpha,\beta,\mu$. The natural way I can think is to minimize some criterion, say, MSE, between $\hat{F}$ and $F_X$.

I have two questions:

  1. Besides MSE, is there any other natural criterion to use? (e.g. some sort of empirical version of Kullback Leibler Divergence?)
  2. Is there an entirely different approach to this estimation question? Is there literature of fitting parametric cdf through ecdf that I need to know?
$\endgroup$

2 Answers 2

4
$\begingroup$

I wouldn't use MSE simply because $\hat{F}$ doesn't have constant variance. If you used a weighted MSE that should do better ... but then $\hat{F}_i$ and $\hat{F}_j$ aren't independent, either.

One possibility if you're trying to make $\hat F$ as close to $F$ in a MSE-like sense as possible is you might try minimizing the Anderson-Darling statistic (since it's like a precision-weighted MSE).

To be honest, I'd be inclined to use likelihood to estimate the parameters, if at all feasible -- but not on the ECDF; you want to deal with the data and construct the likelihood (i.e. look at data values and $f$, not $F$ and $\hat{F}$) if you difference back to what's essentially a histogram$^\dagger$ you should be able to get good ML estimates* from that.

enter image description here

$\hspace{3cm}\to$ 4 5 5 5 6 6 6 7 7 7 7 8 9 9 9 9 9 9 ...

* (if slightly approximate because of the discretization)

$\dagger$ - In case it's not obvious how one obtains the sample size in the case where you only have an ecdf and not $n$, let me be explicit about it. The situation you seem to be dealing with (i.e. as "at a suitably fine grid" suggests) we have a finely discretized continuous distribution, one we might reasonably still treat as if it were continuous if only we had the observations.

Consider that at each jump point, the ecdf must increase by some multiple of $1/n$.

The chance that it will always increase by a multiple of $k/n$ ($2/n$ or $3/n$ say) at every jump point would be extremely small indeed (some simple hand calculations or simulations make the point well enough). So we can - with probability extremely close to 1 - infer the exact $n$ by finding the largest $1/n$ consistent with every change in ecdf.

[On the other hand, with a very small sample at a very coarse grid, so that only a few different values will occur, you could get the increase at each point being $2/n$ or perhaps $3/n$ and so run into problems, particularly if you want standard errors. So if we only had 4 bins and the counts in those bins were 2,6,12,4, we'd get $n$ wrong (it would look like the jumps were multiples of 1/12 rather than 1/24, so we'd think the $n_i$ were 1,3,6,2), and so the standard errors - and in some cases, even the point estimates - could be a fair way off. Fortunately, this isn't the case for this question.]

$\endgroup$
11
  • $\begingroup$ The Anderson-Darling stats is a useful suggestion. Thank you. Re differencing back to the histogram and MLE from there, that did come across my mind, but i wasn't sure whether the differencing will do more harm than the MLE will make up for. But since my grid is suitably fine, I hope differencing will be ok? $\endgroup$
    – qoheleth
    Nov 5, 2014 at 4:33
  • $\begingroup$ To get the ecdf you would have essentially summed a histogram. I'm just suggesting going back one step (reversing that summation back to the discretized data). $\endgroup$
    – Glen_b
    Nov 5, 2014 at 5:44
  • $\begingroup$ That will do no more harm than the original discretization, since it's just getting the discretized data back. Unless you don't know the original sample size (and you should be able to infer that from the ecdf anyway), this is straightforward calculation. $\endgroup$
    – Glen_b
    Nov 5, 2014 at 6:23
  • $\begingroup$ Glen, the unknown sample size seems to be a significant complication and perhaps a barrier to applying ML (especially if standard errors for the parameters are sought). What are your thoughts about handling that? $\endgroup$
    – whuber
    Nov 5, 2014 at 15:42
  • $\begingroup$ @whuber The jumps in ecdf must all be multiples of $1/n$. While it's possible to get situations where counts in each bin are all even, say, with many small bins that's extremely unlikely. So with even fairly moderate sample size, nearly always the largest $1/n$ consistent with the ecdf will be the correct one. When $n$ is large, and we treat the sample as continuous, typically point estimates won't usually be sensitive to the value of $n$ (you'll typically get very similar estimates whether you think $n$ is 300 or 76800, say), ... $\endgroup$
    – Glen_b
    Nov 5, 2014 at 21:57
3
$\begingroup$

You asked about relevant litterature.

If your model is a power law, log-normal, poisson or binomial you should check the work of Clauset et al. "PowerLaw Distribution in Empirical Data" http://arxiv.org/pdf/0706.1062.pdf They describe how to do so by bootstraping, performing fits on the ecdf of the actual data and generated data. You must be cautious when using such approach and follow the procedure described in details in this paper.

An important thing to remember is that you never prove that your data follow a model, you can only disprove it follow a model. p-values give you the closeness of the data and the model, but you can only carefully make conclusions by performing the same test for other likely models that could explain the data.

The authors of the paper also provide a library to perfom the tests for these models (poweRlaw package in R), and partial implementation in other languages.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.