2
$\begingroup$

I am confused about the difference between the nominal and the actual coverage probability.

Say we are trying to estimate the fraction of the current population of the US that has been diagnosed with cancer. We get a sample (e.g. 10,000 individuals) and then estimate this fraction ($f$) and a confidence interval around it $[a,b]$ with $\alpha$ (confidence level) $=0.95$.

At this point, if I understood things correctly, the nominal coverage probability is $\alpha$, i.e. $0.95$.

But what about the actual coverage probability? To me, there is only one true value of $f$, e.g. say $5\%$, and it doesn't make sense to talk about the actual probability that $f$ is within the interval $[a,b]$. It either is or it isn't within $[a,b]$.

What's wrong with this reasoning? And how would one obtain the actual coverage probability in practice?

Improve this question
$\endgroup$
5
  • $\begingroup$ Is your question perhaps answered at stats.stackexchange.com/questions/26450/…? If not, how does it differ? $\endgroup$ – whuber♦ Jul 10 '13 at 17:13
  • $\begingroup$ @whuber. Thanks, I went through that thread, and while interesting (and maybe I am missing something, since I don't fully understand some of the answers), it wasn't clear to me how they addressed the difference between nonimal vs actual coverage probabilities. $\endgroup$ – Amelio Vazquez-Reina Jul 10 '13 at 17:24
  • $\begingroup$ BTW @whuber, I think I have a good understanding of CIs - the idea being that if we conduct repeated experiments, the proportion of times that the given interval contains the true value of the parameter is its confidence level, but I don't know how to make the leap to nominal vs actual cov. probs. $\endgroup$ – Amelio Vazquez-Reina Jul 10 '13 at 17:38
  • 1
    $\begingroup$ That thread discusses the meaning and interpretation of the interval $[a,b]$ covering $f$ and it appears this is exactly the point you are asking about. Also please see the accepted answer at stats.stackexchange.com/questions/6652 (and then browse the other answers). $\endgroup$ – whuber♦ Jul 10 '13 at 17:50
  • 1
    $\begingroup$ The distinction between nominal & actual made in the Wikipedia article is addressed here. $\endgroup$ – Scortchi - Reinstate Monica♦ Feb 11 '14 at 9:44