2
$\begingroup$

In short: I'm wondering if I'm understanding the concept of two-sided "Confidence Interval" via an example?

Details:

Suppose we observe 5 successes in 20 trials and so our observed $p = 5/20$. To obtain a 95% CI for the PARAMETER of this observed $p$, we argue this way:

What the unknown, fixed PARAMETER value of this observed $p$ could be such that out of infinitely many repetitions of a binomial experiment with $20$ trials, $95\%$ of the time we capture that PARAMETER value? Now we take two steps:

First, we find the $p$ of a binomial distribution given 20 trials which strictly allows 2.5% probability for 5 or more successes in 20 trials to happen in large repetitions (the blue binomial distribution). This $p$ can be shown (e.g., by optimization or via Clopper & Pearson) to be "$0.08657147$". This will be the smallest limit value if we were to construct a 95%-coverage uncertainty net to capture the PARAMETER value of the observed $p$. In other words, this "$0.08657147$" is the smallest possible $p$ for a binomial distribution given 20 trials that allows at least 2.5% probability for 5 success out of 20 trials to occur in large repetitions.

Second, we find the $p$ of a binomial distribution given 20 trials which strictly allows 2.5% probability for 5 or less successes in 20 trials to happen in large repetitions (the red dashed binomial distribution). This $p$ can be shown (by optimization or via Clopper & Pearson) to be "$0.49104587$". This will be the largest limit value if we were to construct a 95%-coverage uncertainty net to capture the PARAMETER value of the observed $p$. In other words, this "$0.49104587$" is the largest possible $p$ for a binomial distribution given 20 trials that allows at most 2.5% probability for 5 success out of 20 trials to occur in large repetitions.

enter image description here

Improve this question
$\endgroup$
10
  • $\begingroup$ The interval will differ if you have a one-sided versus a two-sided interval. $\endgroup$
    – Michael R. Chernick
    Jul 28, 2017 at 2:45
  • $\begingroup$ I agree with your second paragraph. The interval you show is described correctly for a two-sided equal tailed interval. The figures you show are confusing. I don't see why you produce two beta curves and also don't understand what the two binomial distributions are for. $\endgroup$
    – Michael R. Chernick
    Jul 28, 2017 at 3:29
  • $\begingroup$ @MichaelChernick, that's the Clopper & Pearson method, that is what the $Beta$ distributions are for. $\endgroup$
    – rnorouzian
    Jul 28, 2017 at 3:39
  • $\begingroup$ I think there is only one distribution for which you determine upper and lower tails. Yes the Clopper-Pearson method is one way to calculate exact binomial probabilities. Also two tailed 95% confidence intervals do not need to be equal tailed. The definition allows say 1% in the lower tail and 4% in the upper (for example) rather than 2.5% in each tail. $\endgroup$
    – Michael R. Chernick
    Jul 28, 2017 at 12:32
  • $\begingroup$ @MichaelChernick, I assume by "there is only one distribution for which you determine upper and lower tails", you 're referring to "Normal Approximation" which due to the symmetry of a normal distribution doesn't require two distribution. But I did not ask about that approximation. Furthermore, generally speaking, one-distribution method has limited generality as many important statistics follow asymmetrical distributions for which an approximation is not known. One thing I do want to know, though, how a $Binomial$ distribution can directly be converted back to a corresponding $Beta$ ... $\endgroup$
    – rnorouzian
    Jul 28, 2017 at 12:55

1 Answer 1

Reset to default
1
$\begingroup$

This image may help your understanding:

Two-sided vs one-sided intervals

Essentially for a two-sided confidence interval, it ranges from an upper to a lower limit, with 2.5% chance on both sides.

A one-sided confidence interval exists when you only look at values above OR below a certain null value (the value that satisfies the null hypothesis). The 95% CI, and the 5% outside the interval, are both above or both below the null value.

Improve this answer
$\endgroup$
1
  • $\begingroup$ You can think of a one-sided intervals as going off to plus or minus infinity (or perhaps some finite bound that makes sense, like 100 for conversion rates). The overlap between two one-sided (1-a)% CIs makes one two-sided 2a% CI where each one-sided interval contributes one bound. $\endgroup$
    – dimitriy
    Aug 4, 2020 at 21:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.