What would be the most sensible way to build a confidence interval if I have no other information than a the number of successes and failures with an experiment ?

Is it possible to make some assumptions and build a confidence interval ?

This is because I have limited information on medical results from different clinics and they provided only the number of patients who did a particular treatment and the number whose treatment was considered successful.

I would like to build some kind of confidence interval to compare them since some had very low number of patients and some had much larger numbers but I do not know anything else than these numbers.

For example, clinic A will say that 95 patients had a success out of 180 total patients for this procedure, while clinic B had 250 successes out of 498 patients.

EDIT: I would like to know the 95% probability interval for the true probability of a success.

  • $\begingroup$ Edited the question $\endgroup$ – BlueTrin Jun 13 '15 at 15:02

Interesting question. Your data is Bernoulli which is a binary distribution with probability of sucess equal to $p$. The simplest thing you can do is compute the probability of success from your sample and use the Central Limit Theorem to arrive at an asymptotic 95% confidence interval. If we denote the sample proportion of success by $\widehat{p}$ then the interval for the true probability $p$ would be

$$\widehat{p}\pm 1.96 \times \sqrt{\frac{\widehat{p} \left(1-\widehat{p} \right)}{n}}$$

I have to emphasize that this is not a probability interval since the true parameter $p$ is considered fixed and does not have a sampling distribution. A long-run interpretation of this interval is that if you gather $100$ samples and compute for each the confidence interval above, then 95 of these will contain the true parameter.

Now you also mentioned that you would like to compare probabilities of success. Let us use the index $1$ and $2$ for the first and second sample respectively. Assuming independent samples, an extension of the above procedure would be to compute the confidence interval

$$\widehat{p_1}-\widehat{p_2}\pm 1.96 \times \sqrt{\frac{\widehat{p}_1 \left(1-\widehat{p}_1 \right)}{n_1}+\frac{\widehat{p}_2 \left(1-\widehat{p}_2 \right)}{n_2}}$$

Notice that this interval now concerns the difference in the population probabilities, i.e. the difference of the true probabilities. The interpretation is precisely the same, neverthless. Gather 100 samples (from each population), compute the confidence intervals and in 95 cases you will find that they contain the population difference. You decide that the population probabilities are not equal if the interval does not contain $0$.

This is just one way to compare population probabilities, though. In practice this method might not be the one which you want to use. The reason is that these intervals are not always very informative. Instead one might want to find an interval for the relative risk $\frac{p_1}{p_2}$ or the odds ratio $\displaystyle{\frac{\frac{p_1}{1-p_1}}{\frac{p_2}{1-p_2}}}$. This is also possible using an asymptotic approximation. If you think that this will be of interest to you, here is a relevant question concerning the relative risk

How to calculate the relative risk based on two independent confidence intervals


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.