# Confidence/credibility intervals for a bernoulli trial

Say we have $$X \sim \text{Bernoulli}(p).$$ I am interested in finding a 95% confidence and credibility interval.

For the credibility, I am assuming a uniform prior, giving me a posterior distribution $$f(p | x) = \text{Beta}(x + 1, 2-x).$$ Therefore we simply take the region of minimal measure such that the area under the distribution is 0.95

However, in the frequentist case for the confidence interval. I am mildly stumped as to how I would proceed. Could I have some insight on the intuition of this problem?

Thank you!

• Just to clarify, you’re looking for a confidence interval for $p$ based on a single sample? Or do you have multiple (possibly iid) draws from this distribution?
– jld
Commented Mar 13, 2018 at 22:47
• @Chaconne We have only 1 sample. So for example 1 coin flip with heads. Commented Mar 13, 2018 at 23:05
• Then a frequentist has nothing to say to you. But why on earth will someone think you will have a frequentist confidence interval from a single sample? Commented Mar 13, 2018 at 23:17
• @ChamberlainFoncha . Indeed, the confidence might not be quite informative, but it does exist nonetheless. I am however interested in understand how one can find this. Commented Mar 13, 2018 at 23:20
• @ChamberlainFoncha just because people usually use a normal approximation does not mean that is all there is. It is really easy to construct a valid Frequentist interval for this problem; just take all values for which the LRT fails to reject.
– guy
Commented Mar 14, 2018 at 1:06

When you only have a single Bernoulli trial, there is little that can be done within the frequentist paradigm. Your maximum-likelihood estimator (MLE) for $p$ is clearly $\hat{p} = X \sim \text{Bern}(p)$. This is an unbiased estimator with unknown variance $\mathbb{V}(\hat{p}) = p(1-p)$. Since you only have a single value in the sample, there is no basis to estimate the variance of the MLE$^\dagger$. Nonetheless, a $(1-\alpha)$-level confidence interval for the parameter $p$ can be constructed as (hap-tip to guy in the comments):

$$\begin{matrix} \text{CI}(1-\alpha) \equiv [\alpha X, 1 - \alpha (1-X)] & & & 0 < \alpha < 1. \end{matrix}$$

This gives an interval estimator for the true probability parameter. It is wider than the Bayesian credibility interval in the question, and thus represents a more robust estimate.

$^\dagger$ The maximum-likelihood estimator of the variance $\mathbb{V}(\hat{p})$ is zero, which is unhelpful.

• Interesting, Under what conditions then can we obtain a confidence interval in the frequentist framework? Commented Mar 13, 2018 at 23:53
• I disagree with this regarding it being impossible to do Frequentist inference. You can get a valid 95% interval by taking [0,.95] if X is zero and [0.05,1] if X is 1, which I assume is what you get by inverting the LRT. I would guess this interval is optimal in some sense (UMPU?).
– guy
Commented Mar 14, 2018 at 0:24
• @rannoudanames UMPU is uniformly most powerful unbiased, but I guess for a CI it should be something like minimal-length unbiased but I’m not sure of the terminology.
– guy
Commented Mar 14, 2018 at 1:00
• @Ben you don’t need a pivot to construct an interval estimate. You can check by direct computation that it has valid coverage, meaning that the infimum over p of the probability of coverage is at least 0.95.
– guy
Commented Mar 14, 2018 at 1:25
• @rannoudanames $x$ is either 0 or 1, so the LR is $p$ when $X=1$ and $1-p$ when $X=0$. For the purposes of the LRT you should interpret $0^0 = 1$ so the denominator is $1$. The LRT then says to reject for $p<k$ when you get a 1 and reject for $p>1-k$ when you get a 0. Set $k=0.05$ to get the interval.
– guy
Commented Mar 14, 2018 at 3:32