3
$\begingroup$

I have a setup where I am making X number of measurements over a particular time frame. These X measurements are success/failure measurements. For example, in the period of an hour I might get 422 successes and 584 failures. Then I calculate the success rate for that hour of measurements being 0.72260273972. Up to what decimal place can I be sure that I am calculating a meaningful number? Is it a function of the number of measurements in that hour or simply the significant figures used in the calculation.

Some help would be much appreciated!

$\endgroup$
4
$\begingroup$

This is the oft-encountered problem of estimating a "confidence interval for a Binomial proportion". There are several different techniques, which vary in their assumptions and are well described on the Wikipedia page.

For all reasonable approaches, the precision (i.e. width of the confidence interval) will depend on

  1. your sample success ratio (i.e. $\hat{p}=\frac{422}{n}$)
  2. the number of measurements (i.e. $n=422+584$)
  3. the desired "confidence level" (i.e. a number $\alpha\in(0,1)$ giving the fractional coverage of the associated interval estimate)

(Note: I put some terms in quotes to try and steer clear of the thorny issues surrounding confidence-intervals vs. credible-intervals, $p$-values, etc.)


Update: At the request of the OP, here is some clarification on how the interval estimate would relate to the numerical precision.

For simplicity, I will assume that "$d$ significant digits" can be interpreted as "$\pm\,5\times 10^{-(d+1)}$", i.e. $$ \hat{p} = 0.722 \implies p = \hat{p} \pm 0.0005 \implies 0.7215 < p < 0.7225 $$ where $\hat{p}$ is the estimate of the true value $p$. So a significance of $d$ digits roughly corresponds to an interval width of $10^{-d}$.

Again for simplicity, assume we use the "normal approximation", so for a given "confidence level" (specified as a "$z$-score") we have interval width $$ \frac{\Delta{p}}{z}=\sqrt{\frac{\hat{p}(1-\hat{p})}{n}} $$ So, for example if we use a coverage probability of 95%, corresponding to $z=1.96$ (roughly 2 standard deviations), then for $\hat{p}=\frac{422}{n}$ and $n=422+584$, we have $$ \Delta{p}\approx 3 \times 10^{-2} = \mathrm{O}[10^{-2}] $$ so, keeping things simple, we might say that there are roughly two significant digits. So while nominally we have $\hat{p}\approx 0.419483101391650$, from these calculations $p\approx 0.42$ would be a reasonable inference (with the $\pm 0.03/2$ implicit in the significant-digit truncation).

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ How does this allow me to know whether 0.72260273972 is a meaningful number or nonsense as opposed to 0.722, where 8 digits were truncated? $\endgroup$ – user2253546 Oct 26 '16 at 2:57
  • $\begingroup$ @user2253546 does the update clarify things? $\endgroup$ – GeoMatt22 Oct 26 '16 at 3:49
  • $\begingroup$ Yes it definitely does. I can see the reasoning behind it. Would you have sources of where this is used? Btw, thank you for the well explained response :) $\endgroup$ – user2253546 Oct 26 '16 at 4:59
  • $\begingroup$ I pieced together this approach from the "standard ingredients" in the Wikipedia links contained in my answer (most of those pages have sources, but no aggregated ref I can give). In terms of "significant digits", this post presents some reasonable guidelines, and this one gives a possible general reference* that is from a pretty official source (*i.e. it probably does not present the particular method I presented here). $\endgroup$ – GeoMatt22 Oct 26 '16 at 5:26
  • $\begingroup$ GUM reference online here. I cannot find a good cite for it on Google Scholar, but this appears similar, and may be the predecessor (NIST/USA vs. BIPM/France). $\endgroup$ – GeoMatt22 Oct 26 '16 at 5:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.