In simulating iterative decoding of low-density parity-check codes there may be (for a certain signal-to-noise-ratio of a noisy channel) for example 10 decoding failures out of $10^6$ trials. The log of probability of decoding failure is plotted against signal-to-noise-ratio.
I quote from the Appendix of MacKay's paper Good Error-Correcting Codes based on Very Sparse Matrices to show how he deals with creating error bars on such plots:
The experiments result in a certain number of block decoding failures $r$ out of a number of trials $n$. We report the maximum likelihood estimate of the block error probability, $\hat{p}=r/n$, and a confidence interval $[p_-,p_+]$, defined thusly: if $r \geq 1$ then $p_{\pm}=\hat{p}\exp(\pm 2\sigma_{\log p})$ where $\sigma_{\log p}=\sqrt{(n-r)/(rn)}$; else if $r=0$ then $p_+=1-\exp(-2/n)$ and $p_-=0$.
I've read the wikipedia page on Binomial proportion confidence interval and a number of helpful answers on this website. Mackay's formula can be derived by approximating the normal approximation to a 2 sigma confidence interval: $\hat{p}\pm z_{1-\alpha/2}\sqrt{\hat{p}(1-\hat{p})/n} = r/n \pm 2\sqrt{r(n-r)/(n^3)}=(r/n)(1\pm2\sqrt{(n-r)/(rn)})$ which is approximately equal to $(r/n)\exp(\pm2\sqrt{(n-r)/(rn)})$ when the argument of the exponential is small.
His notation also suggests he is really doing statistics with $\log p$. His formulas give symmetric error bars on log plots. I don't know whether this is a reasonable goal.
So I have a couple of questions:
Is my derivation the best way to understand what's going on with this formula?
Is it the most appropriate confidence interval?
