Say we have $$ X \sim \text{Bernoulli}(p). $$ I am interested in finding a 95% confidence and credibility interval.

For the credibility, I am assuming a uniform prior, giving me a posterior distribution $$ f(p | x) = \text{Beta}(x + 1, 2-x). $$ Therefore we simply take the region of minimal measure such that the area under the distribution is 0.95

However, in the frequentist case for the confidence interval. I am mildly stumped as to how I would proceed. Could I have some insight on the intuition of this problem?

Thank you!

  • $\begingroup$ minor comment: i think you mean "region of minimum measure" rather than cardinality. Any subset of $\mathbb R$ with positive measure has the same cardinality $\endgroup$ – jld Mar 13 '18 at 19:29
  • $\begingroup$ Yes! Making an edit $\endgroup$ – rannoudanames Mar 13 '18 at 19:41
  • $\begingroup$ Just to clarify, you’re looking for a confidence interval for $p$ based on a single sample? Or do you have multiple (possibly iid) draws from this distribution? $\endgroup$ – jld Mar 13 '18 at 22:47
  • $\begingroup$ @Chaconne We have only 1 sample. So for example 1 coin flip with heads. $\endgroup$ – rannoudanames Mar 13 '18 at 23:05
  • 2
    $\begingroup$ @ChamberlainFoncha just because people usually use a normal approximation does not mean that is all there is. It is really easy to construct a valid Frequentist interval for this problem; just take all values for which the LRT fails to reject. $\endgroup$ – guy Mar 14 '18 at 1:06

When you only have a single Bernoulli trial, there is little that can be done within the frequentist paradigm. Your maximum-likelihood estimator (MLE) for $p$ is clearly $\hat{p} = X \sim \text{Bern}(p)$. This is an unbiased estimator with unknown variance $\mathbb{V}(\hat{p}) = p(1-p)$. Since you only have a single value in the sample, there is no basis to estimate the variance of the MLE$^\dagger$. Nonetheless, a $(1-\alpha)$-level confidence interval for the parameter $p$ can be constructed as (hap-tip to guy in the comments):

$$\begin{matrix} \text{CI}(1-\alpha) \equiv [\alpha X, 1 - \alpha (1-X)] & & & 0 < \alpha < 1. \end{matrix}$$

This gives an interval estimator for the true probability parameter. It is wider than the Bayesian credibility interval in the question, and thus represents a more robust estimate.

$^\dagger$ The maximum-likelihood estimator of the variance $\mathbb{V}(\hat{p})$ is zero, which is unhelpful.

| cite | improve this answer | |
  • $\begingroup$ Interesting, Under what conditions then can we obtain a confidence interval in the frequentist framework? $\endgroup$ – rannoudanames Mar 13 '18 at 23:53
  • $\begingroup$ I disagree with this regarding it being impossible to do Frequentist inference. You can get a valid 95% interval by taking [0,.95] if X is zero and [0.05,1] if X is 1, which I assume is what you get by inverting the LRT. I would guess this interval is optimal in some sense (UMPU?). $\endgroup$ – guy Mar 14 '18 at 0:24
  • 1
    $\begingroup$ @rannoudanames UMPU is uniformly most powerful unbiased, but I guess for a CI it should be something like minimal-length unbiased but I’m not sure of the terminology. $\endgroup$ – guy Mar 14 '18 at 1:00
  • 1
    $\begingroup$ @Ben you don’t need a pivot to construct an interval estimate. You can check by direct computation that it has valid coverage, meaning that the infimum over p of the probability of coverage is at least 0.95. $\endgroup$ – guy Mar 14 '18 at 1:25
  • 1
    $\begingroup$ @rannoudanames $x$ is either 0 or 1, so the LR is $p$ when $X=1$ and $1-p$ when $X=0$. For the purposes of the LRT you should interpret $0^0 = 1$ so the denominator is $1$. The LRT then says to reject for $p<k$ when you get a 1 and reject for $p>1-k$ when you get a 0. Set $k=0.05$ to get the interval. $\endgroup$ – guy Mar 14 '18 at 3:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.