Given $n$ samples from a (continuous) distribution X, the obvious thing to do is sort them, and distribute them equally across $[0,1]$ by taking $(x_{(k)}, (k-1/2)/n)$ as estimates of particular points on the CDF, and doing some sort of interpolation between points, as necessary.

Is this the "right" way to make this estimate? How do I get error bars for the estimated points? It doesn't seem like they'd necessarily be symmetric.

  • $\begingroup$ My initial thought was 'I don't think there is an answer to this question.' Similar to what is written in one of the replies. The idea of a confidence interval about some smooth curve, and then getting narrower as n increases, seems like a cool idea, though. $\endgroup$ – Adam Sep 22 '11 at 8:10
  • $\begingroup$ The usual estimate of the cdf itself is the empirical cdf. $\endgroup$ – Glen_b Feb 9 '15 at 3:12
  • $\begingroup$ I'm interested in this topic, too. I suggest also the book "Extreme Value Theory in Engineering", Academic Press (1988), by Enrique Castillo, where different nonparametric estimates of the cdf are presented. $\endgroup$ – al cliver Jun 30 '15 at 9:38

To get error bars, you can construct a confidence interval around the entire empirical cumulative distribution function (ECDF). This can be done using the Dvoretzky-Kiefer-Wolfowitz inequality. If you want the ECDF to be within $\epsilon$ of the true CDF with confidence $1- \alpha,$ then choose the sample size $n$ using $$n \ge \left( {{1} \over {2 \epsilon^2}} \right) \mathrm{ln} \left({{2} \over {\alpha}} \right)$$

So, for example, if you want the ECDF to be within $0.01$ of the CDF with 95% confidence, we find by plugging in that $$n \ge 18444.4$$ so we select $n=18445.$


In statistics there really is no concept of a 'right' estimate, it's just if the estimate you construct has the properties that you are looking for.

Typically if you are trying to estimate a CDF, you will use the ECDF (Empirical CDF) which is just $ Pr(X < x) = \Sigma_{i=1}^n \mathbb{I}_{x_{(i)} \le x}(x) n^{-1}$. Where $X_{(i)}$ is the $i$th order statistic.

The ECDF has many nice properties such as being strongly consistent (pointwise even) to the CDF.

Since you have a discrete approximation of a continuous distribution you can generate quantiles that can be used for confidence intervals in the usual discrete way.

$inf_x( x : Pr(X <x) \ge \pi) $

Of course there is no reason why a confidence interval should be symmetric so I'm confused by your last statement that I think should be clarified.

  • $\begingroup$ The comment was just that error bars are often reported as one number, as if they were symmetric, unless there is a good reason to expect strong asymmetry. Near the middle of the CDF I expect symmetry. At the edges, I don't have a reason to expect symmetry. $\endgroup$ – wnoise Sep 22 '11 at 20:10
  • $\begingroup$ This directly gives estimates everywhere, rather than just at the samples. Between the samples these make some sense, but a bit less at the samples. For a continuous distribution, there is essentially no distinction between $<$ and $\leq$. Symmetry consideration suggest "k-1/2" rather than k-1 or k at the points. This should have very similar properties in the $n \rightarrow \infty$ limit, but seems much saner for low n. The median of an odd number of points is an estimator for an x such that $Pr[X < x] = 1/2$, not some offset to 1/2 varying with n. $\endgroup$ – wnoise Sep 28 '11 at 4:22
  • 1
    $\begingroup$ I'm afraid I don't know the "usual discrete way" of generating these confidence intervals, nor does your $\inf$ notation make sense to me. $\endgroup$ – wnoise Sep 28 '11 at 4:24
  • $\begingroup$ the inf notation is saying the smallest value of x in the set of x such that $Pr( X < x ) \ge \pi$ $\endgroup$ – Jonathan Lisic Oct 2 '11 at 0:16

You could always use a kernel density estimator (which would also give the c.d.f. as the weighted sum of the component c.d.f.s). You could then get error bars by bootstrapping the available data. This would be pretty simple to implement and would give nice, well-behaved smooth c.d.f.s with error bars.

  • 1
    $\begingroup$ The KDE doesn't seem to add anything to just bootstrapping with the Empirical CDF. $\endgroup$ – wnoise Sep 23 '11 at 18:53
  • 3
    $\begingroup$ Smoothness and differentiability? It achieves the "interpolation" between points, but in a more probabilistic manner than direct interpolation. If you don't need a smooth c.d.f. then the bootstrapped empirical CDF sounds as good an approach as any other. $\endgroup$ – Dikran Marsupial Sep 23 '11 at 19:02

In a bayesian approach, you could use a Dirichlet Process (DP) to estimate the PDF and then integrate it. What you are trying to do is to estimate the function based on samples at certain values. The DP approach allows you to incorporate a smoothness assumption, which is useful because you will often prefer a solution that is differentiable than one that looks like a staircase. The outcome of your analysis is then a distribution over functions, which in particular gives you a mean function, and some error bars on it.

The following book has a nice chapter on Dirichlet processes: O'Hagan, A. and Forster, J. J. (2004). Bayesian Inference, 2nd edition, volume 2B of "Kendall's Advanced Theory of Statistics". Arnold, London.

  • $\begingroup$ I'm not interested in the PDF, just the CDF, so the smoothness properties of the PDF don't matter. (And actually I have enough points that just about any smoothing procedure works quite well except near the ends.) What I actually want to do is fit curves to the CDF. For that purpose, I'd like to be able to treat the data as estimates to the CDF in some consistent way, with error bars for weights. $\endgroup$ – wnoise Sep 22 '11 at 20:13
  • 1
    $\begingroup$ @wnoise you get error bars on the CDF with that method, that's the point I was trying to make. If you don't want to integrate the PDF and work directly on the CDF, you can use a gaussian process on your data points, pretty much like you suggested. The gaussian process will then yield the most likely interpolant and its error bars. $\endgroup$ – yannick Sep 22 '11 at 22:59
  • $\begingroup$ Why Gaussian error bars though? I'd think that something like beta(k, n-k) error bars on the point at $x_{(k)}$ would be appropriate. And of course it really seems like adjacent points are correlated... $\endgroup$ – wnoise Sep 23 '11 at 19:12
  • $\begingroup$ what do you mean by gaussian error bars? I must add that what I meant was truly confidence interval, and not error bars. And the GP accounts for correlation in neighboring points, that's the whole point of using it. $\endgroup$ – yannick Sep 24 '11 at 4:09
  • $\begingroup$ For the Beta, that's actually why I was alluding to the Dirichlet process, which is a generalization to functions of the beta distribution. $\endgroup$ – yannick Sep 24 '11 at 4:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.