I'm trying to fully understand the confidence interval formal given on this site:

$$\hat{\mu}\pm z_{1-\alpha/2}\sqrt{\frac{\hat{\mu}(1-\hat{\mu})}{n}}$$

so I can reproduce the same type of intervals for my own data. But I don't quite understand what the parameters such as $\alpha$ and $Z$ mean. I'm guessing they're related to defining a 95% confidence interval if your data were distributed normally.

Can explain to me how that formula works or a reference I can find a description of the formula used?

  • 2
    $\begingroup$ @user4127 Here is a related question that might be helpful. $\endgroup$
    – chl
    Commented Apr 12, 2011 at 7:48

2 Answers 2


If $\hat{\mu}$ is the mean error rate computed averaging $N$ error rates from different $N$ tests, an explanation could be:

Let $X$ be the number of errors on $N$ tests, so $X$ is a binomial distributed random variable with mean $N\hat{\mu}$ and variance $N\hat{\mu}(1-\hat{\mu})$ (it is sum of $N$ Bernoulli random variables).

Thus $X/N\sim Bin\bigg(\hat{\mu},\frac{\hat{\mu}(1-\hat{\mu})}{N}\bigg)$.

By the central limit theorem it could be approximated to a normal random variable with same mean and variance. Then you can compute the $\alpha$ confidence interval with:

$$P\bigg(-z_{1-\alpha/2}\leq\frac{\mu-\hat{\mu}}{\sqrt{\hat{\mu}(1-\hat{\mu})/N}}\leq z_{1-\alpha/2}\bigg) = 1 - \alpha$$

Bibliography: It is similar to estimate a confidence interval for accuracy using a $N$ values test set in a classification problem. You should take a look to P.N. Tan, M. Steinbach, V. Kumar Introduction to Data Mining. Addison Wesley, 2006.

  • $\begingroup$ Thanks for the explanation. So then it seems that I have interpreted their results incorrectly as I thought they had run their experiment multiple times over the entire dataset. I realize now that the confidence interval is based on a single experiment over the dataset. $\endgroup$ Commented Apr 12, 2011 at 15:28
  • $\begingroup$ Yes it is. I don't know if all datasets are the same cardinality ($N=50000$). By the way at least you can computer your own confidence interval for your data. $\endgroup$
    – Simone
    Commented Apr 12, 2011 at 17:56

Answering the following part of your question:

I don't quite understand what the parameters such as alpha and Z mean

$\alpha$ is the parameter that defines the confidence level of the interval. Specifically, the confidence level will be $100(1-\alpha)$%, so to get a 95% confidence interval, set $\alpha=0.05$.

$Z$ is a reference to the normal distribution, and in this case $z_q$ means its $q$-th quantile, that is the value for which $P(Z < z_q) = q$, where $Z$ is the standard normal distribution. This can be looked up in tables or calculated by computers. For example, when $\alpha=0.05$, the formula needs the 0.975-th quantile, that is the value which exceeds 97.5% of the normal distribution. Its value is $z_{0.975}=1.96$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.