Inference with only left-censored data

Question

Suppose I have a data set that is only left-censored data, ex:

<5, <5, <5, <10, <10, <10

A technique to handle left-censored data is the Kaplan Meier estimate, see page 5 of this excellent article:

One problem I will run into is that the KM method relies on some data to have been fully observed. For by above dataset, I have only censorships, so the KM method will return $\hat{F(t)} = 1, \forall t$.

This feels wrong to be, as I feel my intuition can give a better inference. For example, looking at the above data, I know that if an individual enters the study, the have more than a 50% chance of having the event of interest occur before time 5.

What statistical options do I have if all my data is left censored?

Additional thoughts

Conversely, if all my data is right-censored, I can't make a similar argument as above. The above argument works because I know $T \in [0,5]$.

Answer 1

How much you can conclude depends on the assumptions you are willing to make about the underlying distribution. Even with no assumptions, though, you can still conclude something, although it might not be a whole lot. For example, the sample data are not plausibly consistent with an underlying distribution whose mean is $100$ and standard deviation is $10$: Chebyshev's inequality tells us these data must have extremely low likelihood in that case.

The most general answer will be given in the form of a p-box. This is a simple device to describe a range of distribution functions. It consists of two graphs, one lying above the other, which thereby determine a region $\mathcal P$ in Cartesian coordinates. Any CDF whose graph would lie entirely within $\mathcal P$ is an element of that p-box.

As an example, let's construct a (nonparametric) 90%-confidence p-box for the sample data, $(\lt 5, \lt 5, \lt 5, \lt 10, \lt 10, \lt 10)$. This p-box should be as "small as possible" (in some sense) while containing any distribution law $F$ for which the probability of the data, given $F$, is at least $1 - 90/100 = 1/10$. Let us (temporarily) call these the "plausible laws."

"As small as possible" can be rigorously defined. Let $\mathbb{P}$ be the set of all distribution functions. Associated with any subset $\Omega\subset \mathbb{P}$ is the set $C(\Omega)$ consisting of all points $(x,y)$ such that $x$ is a real number and there exist $F,G\in \Omega$ with $F(x) \le y \le G(x)$. Letting $\Omega$ be the set of plausible laws, we take the p-box to be $C(\Omega)$.

These data, which have no quantified results at all, tell us just two things about the plausible laws. When a sample $X_1, X_2, \ldots, X_6$ is taken from such a law $F$, then (1) it should be unlikely that one or more of the $X_i$ equal $10$ or greater and (2) it should be unlikely that four or more of the $X_i$ should equal $5$ or greater. These are binomial events governed by the parameters $p=F(10)$ and $q=F(5)$, respectively. Interpreting "unlikely" as being a chance of $100-90 = 10\%$ or less, these criteria uniquely determine $p$ and $q$.

For these data, $p = (1/10)^{1/6} \approx 0.681$ and $q \approx 0.201$.

The p-box therefore contains the graphs of all distribution laws $F$ for which (1) $q \le F(5) \le 1$ and (2) $p \le F(10) \le 1$. A part of this p-box (between $x=0$ and $x=15$) is shaded in gray:

Within this p-box I have drawn the graphs of some distributions that come close to its boundary. All three are two-parameter Exponential distributions. The dark red one is determined by the two corners it passes through. The other two (dashed gray ones) show a distribution with a large standard deviation and one with a small standard deviation.

These illustrations should make it clear that if you wished to construct a parametric p-box encompassing only two-parameter Exponential distributions, then it would exclude the region between the thick red curve and the lower line segment between $x=5$ and $x=10$, but it would have to include everything else. We don't necessarily achieve a lot by making a strong parametric assumption in this example.

What happens when the amount of data increases? Suppose as you collect more data that all results continue to be either $\lt 5$ or $\lt 10$ and that their proportions remain roughly 1:1. Then the value of $p$ approaches $1.0$--there can't be much of a chance of observing anything greater than $10$--and the value of $q$ approaches $1/2$, the proportion of $\lt 5$s in the population. Making an Exponential distribution assumption allows you to conclude a little more: the chance of observing anything above $5$ grows vanishingly small, too. Here is the situation upon observing $3000$ values of "$\lt 5$" and $3000$ values of "$\lt 10$":

Nevertheless, there is a huge set of possible distributions consistent with such data, even when the dataset becomes enormous. Intuitively this should be clear: for example, any distribution supported on a set of numbers less than $5$ will be consistent with such data and its graph will fit within this p-box.

This approach to data analysis is not as powerful as, say, a Bayesian analysis. However, it requires minimal assumptions: we only suppose that the data are identically and independently distributed. There is no need to adopt a prior and justify your choice of it, for instance. (Such checking is rather difficult when all the data are censored!) If the p-box is sufficiently narrow to let you draw a firm conclusion, then you're done and you know your conclusion is extremely robust. Otherwise, the p-box can be a useful tool to help you explore the impact of potential additional assumptions on your conclusions.

---

Here is some R code to play with.

k1 <- 3; k2 <- 3     # Counts
x1 <- 5; x2 <- 10    # Quantitation limits
confidence <- 90/100
#
# Find p and q.
#
x <- c(rep(x1, k1), rep(x2, k2))
n <- length(x)
alpha <- 1 - confidence
p <- uniroot(function(p) pbinom(n-1, n, p) - confidence, c(0,1))$root
q <- uniroot(function(p) pbinom(k1-1, n, p) - confidence, c(0,1))$root
#
# Plot the p-box.
#
plot(c(0, 15), c(0, 1), type="n", bty="n", xlab="X", ylab="Probability")
rect(0, 0, x1, 1, col=gray(.975), border=NA)
rect(x1, q, x2, 1, col=gray(.975), border=NA)
rect(x2, p, 15, 1, col=gray(.975), border=NA)
#
# Show a parametric solution.
#
F.inv <- qexp; F <- pexp            # Exponential distribution family
F.p <- F.inv(p); F.q <- F.inv(q)
sigma <- (x2 - x1) / (F.p - F.q)    # Scale parameter
mu <- x2 - sigma*F.p                # Offset parameter
curve(F((x - mu)/sigma), add=TRUE, col="Red", lwd=2)
f <- stepfun(c(x1, x2), c(0, q, p)) # Lower limit of the p-box
plot(f, add=TRUE, verticals=FALSE)
curve(1 + 0*x, add=TRUE)            # Upper limit of the p-box

Answer 2

I can think of one solution, but I'm open to others.

In the paper linked, the use the formula:

$$F(Z_i) = \prod_{i=1}^j \frac{R_i - n_i}{R_i} $$

The term being multiplied, $\frac{R_i - n_i}{R_i}$, is a pointwise estimate of the rate at time $i$. For low $R_i$ size, or when $n_i = R_i$ or $n_i = 0$, this produces somewhat pathological results (see my complaints above). Instead we can treat this estimation as a Bayesian beta-binomial problem: Given prior $Beta(\alpha, \beta)$, we observe $n_i$ deaths in $R_i$ opportunities and update our prior to $Beta(\alpha + n_i, \beta + R_i - n_i)$. (I suggest keep $\alpha, \beta$ very low as to not introduce a strong bias in the final product).

We then sample from each posterior conversion rate and form the product:

$$F(Z_i) = \prod_{i=1}^j (1 - S_i) $$

where $S_i \sim Beta(\alpha + n_i, \beta + R_i - n_i)$. Doing this many times, we can reconstruct the posterior CDF.

Why does this work? Even if $n_i = 0$, $S_i$ is still slightly greater than 0, though not too much greater. If I have time later this week, I'll do some simulations.