5
$\begingroup$

Confidence intervals for binomial proportions have irregular coverage over the range of possible population parameters (e.g. see Brown et al. 2001 <Link>). How can I formally and usefully describe the properties of the confidence intervals?

Say I toss a coin ten times and obtain seven heads. Are the following statements accurate?

For the Clopper-Pearson method:
The interval 0.3475–0.9333 has been generated by a method that will on at least 95% of occasions, for any true population proportion, contain the true population proportion. The long-run frequency with which this method would yield confidence intervals containing the true population proportion pertaining to this particular experiment is at least 95%.

For the Wilson's scores method:
The interval 0.3968–0.8922 has been generated by a method that will on 95% of occasions, averaged over all population proportions, contain the true population proportion. The long-run frequency with which this method would yield confidence intervals containing the true population proportion pertaining to this particular experiment may be more or less than 95%.

Improve this question
$\endgroup$
2
  • $\begingroup$ Those statements sound fine to me. $\endgroup$
    – onestop
    Jan 12, 2011 at 8:25
  • $\begingroup$ I do not think that the second statement is correct, because it is not defined what "average over all population proportions" means. Actually this formulation makes an implicit assumption about an a priori probability of the true value (assumed to be uniform). This is essentially a Bayesian approach and, as you make an a priori assumption anyway, the use of a Bayesian HPD interval makes more sense (incidentally it even has better coverage probability than the Wilson interval). $\endgroup$
    – cdalitz
    Nov 12, 2019 at 10:25

2 Answers 2

Reset to default
4
$\begingroup$

You ask how to 'formally and usefully' present your conclusions

Formally: Your answer is an accurate summary of some of the results from Brown et al. as I understand them. (I note you do not offer their preferred small n method).

Usefully: I wonder who you audience is. For professional statisticians, you could state your two intervals directly with only citations to the original papers - no further exposition needed. For an applied audience, you would surely rather pick an interval on whatever substantive grounds you (or they) have, e.g. a preference for conservative coverage or good behavior for small proportions, etc., and just present that interval alone, noting its nominal and perhaps also its actual coverage much as you do above, perhaps with a footnote to the effect that other intervals are possible.

As it stands you offer a choice of intervals but not much explicit guidance for an applied audience to make use of that information. In short, for that sort of audience I would suggest either more information about the implications of choice of interval. Or less!

Improve this answer
$\endgroup$
3
  • $\begingroup$ I’m writing a paper for a broad audience an wish to contrast those statements with the equivalent statement for the interval containing 95% of the population proportions that are likely to yield the observed number of successes. That interval is easily accessible from the cumulative beta function, Beta(x+1,n-x+1), and has exact coverage given the observed successes. It is fully frequentist, being a fiducial interval and is at the same time a Bayesian interval with a uniform prior. (Ross 2003, <www.computersinbiologyandmedicine.com/article/S0010-4825(03)00019-2/abstract> $\endgroup$
    – Michael Lew
    Jan 16, 2011 at 5:48
  • $\begingroup$ Then I can see why you don't mention the Jeffreys interval: Although I can't access the article itself, from the abstract it seems that Ross 2003 offers a Jeffreys' interval with a Beta(1,1) prior, rather than the same interval with a Beta(.5,.5) prior that Brown et al. examined. It would seem odd if the coverage properties were wildly different for these two, but without the article it's hard to say. $\endgroup$
    – conjugateprior
    Jan 16, 2011 at 11:55
  • $\begingroup$ The coverage properties with a uniform prior and the Jeffrey's prior are indeed very similar when expressed over all possible population parameter values. However, the uniform is superior when the coverage is expressed as conditional on the observed proportion. In that case it is exact. Not 'exact' in the Clopper-Pearson sense, by exactly exact. The Jeffries prior is not quite as good. It is important to note that intervals from the method do not have to be interpreted in the Bayesian manner, as it is entirely derivable as a fiducial interval. $\endgroup$
    – Michael Lew
    Jan 16, 2011 at 22:05
0
$\begingroup$

When using confidence intervals, my response is always as follows:

In repeated sampling % of intervals so constructed will contain _. Thus, we have % confidence that the true lies within __-_.

One example is

In repeated sampling, 95% of all intervals so constructed will contain Mu, the true population mean. Thus we have 95% confidence that the population GPA lies between 2.72 and 3.07.

I don't see anything particularly wrong with your response, except I would imagine the last sentence should be 95%, not more or less, or at least.

Improve this answer
$\endgroup$
1
  • 2
    $\begingroup$ The issue here lies in the distinction between actual coverage and nominal coverage. You seem to assume the two are equal, but for most discrete distributions they are not. Worse, the actual coverage depends on the proportion one is trying to estimate! Before considering the question further, you might want to research the two methods @Michael Lew mentions. $\endgroup$
    – whuber ♦
    Jan 13, 2011 at 23:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.