10
$\begingroup$

Given $n$ samples from a (continuous) distribution X, the obvious thing to do is sort them, and distribute them equally across $[0,1]$ by taking $(x_{(k)}, (k-1/2)/n)$ as estimates of particular points on the CDF, and doing some sort of interpolation between points, as necessary.

Is this the "right" way to make this estimate? How do I get error bars for the estimated points? It doesn't seem like they'd necessarily be symmetric.

Improve this question
$\endgroup$
3
  • $\begingroup$ My initial thought was 'I don't think there is an answer to this question.' Similar to what is written in one of the replies. The idea of a confidence interval about some smooth curve, and then getting narrower as n increases, seems like a cool idea, though. $\endgroup$
    – Adam
    Sep 22, 2011 at 8:10
  • 2
    $\begingroup$ The usual estimate of the cdf itself is the empirical cdf. $\endgroup$
    – Glen_b
    Feb 9, 2015 at 3:12
  • 1
    $\begingroup$ I'm interested in this topic, too. I suggest also the book "Extreme Value Theory in Engineering", Academic Press (1988), by Enrique Castillo, where different nonparametric estimates of the cdf are presented. $\endgroup$
    – al cliver
    Jun 30, 2015 at 9:38

4 Answers 4

Reset to default
7
$\begingroup$

To get error bars, you can construct a confidence interval around the entire empirical cumulative distribution function (ECDF). This can be done using the Dvoretzky-Kiefer-Wolfowitz inequality. If you want the ECDF to be within $\epsilon$ of the true CDF with confidence $1- \alpha,$ then choose the sample size $n$ using $$n \ge \left( {{1} \over {2 \epsilon^2}} \right) \mathrm{ln} \left({{2} \over {\alpha}} \right)$$

So, for example, if you want the ECDF to be within $0.01$ of the CDF with 95% confidence, we find by plugging in that $$n \ge 18444.4$$ so we select $n=18445.$

Improve this answer
$\endgroup$
10
$\begingroup$

In statistics there really is no concept of a 'right' estimate, it's just if the estimate you construct has the properties that you are looking for.

Typically if you are trying to estimate a CDF, you will use the ECDF (Empirical CDF) which is just $ Pr(X < x) = \Sigma_{i=1}^n \mathbb{I}_{x_{(i)} \le x}(x) n^{-1}$. Where $X_{(i)}$ is the $i$th order statistic.

The ECDF has many nice properties such as being strongly consistent (pointwise even) to the CDF.

Since you have a discrete approximation of a continuous distribution you can generate quantiles that can be used for confidence intervals in the usual discrete way.

$inf_x( x : Pr(X <x) \ge \pi) $

Of course there is no reason why a confidence interval should be symmetric so I'm confused by your last statement that I think should be clarified.

Improve this answer
$\endgroup$
4
  • $\begingroup$ The comment was just that error bars are often reported as one number, as if they were symmetric, unless there is a good reason to expect strong asymmetry. Near the middle of the CDF I expect symmetry. At the edges, I don't have a reason to expect symmetry. $\endgroup$
    – wnoise
    Sep 22, 2011 at 20:10
  • $\begingroup$ This directly gives estimates everywhere, rather than just at the samples. Between the samples these make some sense, but a bit less at the samples. For a continuous distribution, there is essentially no distinction between $<$ and $\leq$. Symmetry consideration suggest "k-1/2" rather than k-1 or k at the points. This should have very similar properties in the $n \rightarrow \infty$ limit, but seems much saner for low n. The median of an odd number of points is an estimator for an x such that $Pr[X < x] = 1/2$, not some offset to 1/2 varying with n. $\endgroup$
    – wnoise
    Sep 28, 2011 at 4:22
  • 1
    $\begingroup$ I'm afraid I don't know the "usual discrete way" of generating these confidence intervals, nor does your $\inf$ notation make sense to me. $\endgroup$
    – wnoise
    Sep 28, 2011 at 4:24
  • $\begingroup$ the inf notation is saying the smallest value of x in the set of x such that $Pr( X < x ) \ge \pi$ $\endgroup$
    – Jonathan Lisic
    Oct 2, 2011 at 0:16
3
$\begingroup$

You could always use a kernel density estimator (which would also give the c.d.f. as the weighted sum of the component c.d.f.s). You could then get error bars by bootstrapping the available data. This would be pretty simple to implement and would give nice, well-behaved smooth c.d.f.s with error bars.

Improve this answer
$\endgroup$
2
  • 2
    $\begingroup$ The KDE doesn't seem to add anything to just bootstrapping with the Empirical CDF. $\endgroup$
    – wnoise
    Sep 23, 2011 at 18:53
  • 3
    $\begingroup$ Smoothness and differentiability? It achieves the "interpolation" between points, but in a more probabilistic manner than direct interpolation. If you don't need a smooth c.d.f. then the bootstrapped empirical CDF sounds as good an approach as any other. $\endgroup$
    – Dikran Marsupial
    Sep 23, 2011 at 19:02
2
$\begingroup$

In a bayesian approach, you could use a Dirichlet Process (DP) to estimate the PDF and then integrate it. What you are trying to do is to estimate the function based on samples at certain values. The DP approach allows you to incorporate a smoothness assumption, which is useful because you will often prefer a solution that is differentiable than one that looks like a staircase. The outcome of your analysis is then a distribution over functions, which in particular gives you a mean function, and some error bars on it.

The following book has a nice chapter on Dirichlet processes: O'Hagan, A. and Forster, J. J. (2004). Bayesian Inference, 2nd edition, volume 2B of "Kendall's Advanced Theory of Statistics". Arnold, London.

Improve this answer
$\endgroup$
5
  • $\begingroup$ I'm not interested in the PDF, just the CDF, so the smoothness properties of the PDF don't matter. (And actually I have enough points that just about any smoothing procedure works quite well except near the ends.) What I actually want to do is fit curves to the CDF. For that purpose, I'd like to be able to treat the data as estimates to the CDF in some consistent way, with error bars for weights. $\endgroup$
    – wnoise
    Sep 22, 2011 at 20:13
  • 1
    $\begingroup$ @wnoise you get error bars on the CDF with that method, that's the point I was trying to make. If you don't want to integrate the PDF and work directly on the CDF, you can use a gaussian process on your data points, pretty much like you suggested. The gaussian process will then yield the most likely interpolant and its error bars. $\endgroup$
    – yannick
    Sep 22, 2011 at 22:59
  • $\begingroup$ Why Gaussian error bars though? I'd think that something like beta(k, n-k) error bars on the point at $x_{(k)}$ would be appropriate. And of course it really seems like adjacent points are correlated... $\endgroup$
    – wnoise
    Sep 23, 2011 at 19:12
  • $\begingroup$ what do you mean by gaussian error bars? I must add that what I meant was truly confidence interval, and not error bars. And the GP accounts for correlation in neighboring points, that's the whole point of using it. $\endgroup$
    – yannick
    Sep 24, 2011 at 4:09
  • $\begingroup$ For the Beta, that's actually why I was alluding to the Dirichlet process, which is a generalization to functions of the beta distribution. $\endgroup$
    – yannick
    Sep 24, 2011 at 4:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.