I would like to calculate a confidence interval for the probability of a process succeeding after a given amount of time by running the process $n$ independent times and recording the time at which each run succeeded. Essentially, I would like to add confidence intervals to a plot like this:
I think this can be modelled as the probability of $n$ coins having ever come up heads after $t=1,2,\ldots$ flips. If I had $n$ independent runs of each different number of flips, I would expect to model the confidence interval at each number of flips as a binomial proportion confidence interval, e.g.,
- flip all coins ($n_1 = n$) once each (i.e., $t=1$) and count each coin that came up heads ($s_1$),
- flip all coins ($n_2 = n$) twice each (i.e., $t=2$) and count each coin that came up heads on either flip ($s_2$),
- etc.
and then calculate the Clopper–Pearson interval for each $s_t/n$.
But in this scenario the subsequent flips aren't independent and it is equivalent to,
- flipping all coins ($n_1 = n$) once (i.e., $t=1$) and counting the number of heads ($s_1$),
- flipping only the coins that weren't heads ($n_2 = n-s_1$) again (i.e., $t=2$) and counting the cumulative number of heads ($s_1 + s_2$),
- etc.
In this cumulative scenario, I could still calculate the Clopper-Pearson interval at each $t$ from $\frac{\sum_{i=1}^t s_i}{n}$, but is that correct? It seems like it ignores the time relationship...
I'm sure this is a common problem and I am just thinking about it wrong or not using the right terms/language, so any direction is appreciated! If it is relevant, I am actually interested in the problem when $t$ is a real valued number (e.g., the amount of time a process has been running).