Let $P$ be an unknown distribution on $(-\infty,\infty)$. Let $X_1,\ldots,X_n$ be an iid sample from $P$. Let $c_1,\ldots,c_n\in(-\infty,\infty)$ be a known set of constants. We observe $Y_1,\ldots,Y_n$, where $Y_i = 1(X_i < c_i)$. (That is, $Y_i=1$ if $X_i<c_i$ and is $0$ otherwise.) I'm looking for some "reasonable" estimators of the distribution $P$.

And what do I mean by "reasonable"? Suppose, for simplicity, that $P$ is supported on [0,1], and the $c_i$ are an iid sample from Uniform[0,1]. $P$ probably has atoms, but we can ignore that if it invites more answers.

Call $\hat{P}_n$ a "reasonable" estimator of $P$ if, with probability $1$ over the selection of the $c_i$'s, $\hat{P}_n$ converges to $P$ in distribution.

NOTE: We can actually consider the $c_i$ as design points that can be chosen by the experimenter. That seems like a separate issue, and I don't want to complicate the question. But if you have an estimator that requires a particular set of $c_i$'s, that's fine too.

  • $\begingroup$ "Reasonable" implies you have a loss function in mind, for otherwise this question is unanswerable. What is it? And are there any assumptions you can make about $P$ at all? $\endgroup$
    – whuber
    Jun 6, 2013 at 16:24
  • $\begingroup$ Good point @whuber. I've tried to clarify what I mean. $\endgroup$
    – DavidR
    Jun 6, 2013 at 17:20

1 Answer 1


You could try to directly estimate the CDF via a binomial rate smoother ? Here is an idealized example for x stemming from a normal distribution:

   ci = seq(from=-3,to=3,length=500)
   X = rnorm(500)
   Y = rep(NA, 500)
   for (i in 1:500) Y[i] = as.numeric(X[i] < ci[i] )
   plot(ci,Y, type="s")
   fit=gam(Y~s(ci), family=binomial(link="logit"))
   plot(fit, trans=inv.logit, shade = TRUE)

enter image description here

To enforce monotonic behavior, in the above example, change the code to:

fitMonotone=scam(Y~s(ci,bs="mpi"), family=binomial(link="logit"))

InvLogit = function(x, SCALE=TRUE) {
  if (SCALE) x = x -mean(x)

plot(fitMonotone, trans=InvLogit, shade = TRUE)

enter image description here

  • $\begingroup$ Yes, can you explain more what a binomial rate smoother does or post a helpful link? $\endgroup$
    – soakley
    Jun 8, 2013 at 21:01
  • $\begingroup$ Is it possible that the CDF estimate won't itself be a valid CDF? In particular, it seems to me that the estimator could occasionally decrease as c increases (at least for a finite sample of c's). $\endgroup$
    – DavidR
    Jun 20, 2013 at 19:22
  • $\begingroup$ Great point ! My oversight; indeed we need to enforce a monotonic constraint. Luckily, the library scam which builds upon mgcv allows to incorporate "shape constraints" such as monotonicity, convexity, etc. I have added new code above $\endgroup$ Jun 22, 2013 at 8:04
  • $\begingroup$ Looks good to me -- thanks! I've been wondering what a nonparametric maximum likelihood estimator would like here, similar in spirit to the Kaplan-Meier estimator for right censored data. $\endgroup$
    – DavidR
    Jun 28, 2013 at 20:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.