2
$\begingroup$

Let's say that I want to compute confidence interval of mean for a purity of crystal. I know for fact that purity of any chemical substance cannot exceed 100%. How can I construct confidence interval of mean, which a statistic that has upper, or lower bound?

Assume the following:

  1. sample size = 21, std = 3, mean = 99.1
  2. samples are normally distributed
  3. upper bound of mean is 100

With Scipy, I can construct its 95% confidence interval like this:

stats.t.interval(1 - 0.05, 21 - 1, loc=99.1, scale= 3 / np.sqrt(21))
>>> (97.73441637228476, 100.46558362771523)

The calculated upper bound for the confidence interval of mean exceeds 100, which is not physically possible in real life.

How do I deliver my conclusion in this case? Is truncating my interval above 100 good, like this?

>>> (97.73441637228476, 100)

Or is there any special modification that I need to make?

$\endgroup$
1
  • $\begingroup$ I'd try a logistic regression or beta regression. See this post. $\endgroup$ Jun 30, 2019 at 9:17

4 Answers 4

1
$\begingroup$

If you want to construct a proper confidence interval, one way to do it would be to assume another distribution than the normal. Indeed the normal distribution has unbounded support, but in your problem, your support is bounded between 0% and 100%.

$\endgroup$
1
$\begingroup$

Would you take a bayesian answer? You get a credible interval instead of a confidence interval, which is a different beast. But on the plus side it's very natural to have a prior that only has support from 0 to 1, then model each observation as coming from a truncated normal centered around the characteristic mean purity. If these seem like reasonable assumptions given your knowledge about crystals then the posterior will be a similarly good summary for talking about the crystal purity.

$\endgroup$
1
$\begingroup$

IIUC, your samples are not normally distributed (contrary to your point 2) but maybe rather according to a truncated normal. So you should use a confidence interval implementation for the truncated normal (or whatever distribution you deem most fitting).

$\endgroup$
0
$\begingroup$

This is a well-known problem with utilizing the Wald confidence interval for binomial distributions. It's typically referred to as "overshoot." There are many alternative to Wald confidence intervals that avoid overshoot and also have better nominal coverage for binomial distributions.

I'd suggest the Wilson score interval but there are other alternatives. This article provides a detailed comparison of the Wald and Wilson score interval. You'll find the relevant formula in the Wikipedia entry as well as lots of references on binomial confidence intervals that you might find helpful.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.