0
$\begingroup$

In a first course to statistics, confidence interval calculation using the Central Limit Theorem is introduced.

  1. Are confidence intervals sensible in practice? That is, given a 99% confidence interval $[a,b]$ for the true mean $\mu$ of a quantity $Q$ using a sample of size $n$, and assuming that data collection was done with enough care, has there been a good track record* for whether one can reliably say that $\mu \in [a,b]$?
  2. After accounting for the error from Central Limit Theorem**, can the probability that $\mu \in [a,b]$ be calculated in practice? One way this can happen is that $\mu \in [a,b]$ holds with probability at least $1-\delta$, assuming that one samples at least $n \ge N_\delta$ points. Then is such an estimate for $N_\delta$ of any practical significance, or is it much larger than the value of $n$ typically required for a decent inference in practice?

*The questions above are somewhat vague, but I think they are important questions. I'd be pleased to hear your thoughts, even if it is only a specific interpretation of the above. Restricting the above questions of "reliability" to specific types of inferences is also welcome (e.g. medical diagnosis, party affiliation, presidential rating, energy of a particle, ...)

**One version of the Berry-Essen theorem quantifies such an error. Suppose $X_1, \cdots X_n$ are i.i.d. samples drawn from a distribution with the first three moments finite and equal to $0, \sigma^2, \rho$. Let $F_n(x)$ be the CDF of the rescaled sample mean $\frac{\sqrt{n}}{\sigma} \cdot \frac{X_1 + \cdots + X_n}{n}$, and $\Phi(x)$ be the CDF of the standard normal. Then there is a constant $C > 0$ such that (By Shevtsova 2011, one may use $C = 0.5$), for any $x$ and $n$, $$| F_n(x) - \Phi(x) | < \frac{C \rho}{\sigma^3 \sqrt{n}} $$ One can use this to bound the error in estimating tail sums with $\Phi$. But we need to estimate $\sigma$ and $\rho$, which involves another statistical inference. Being mostly new to statistics, I personally don't know if reliably estimating $\sigma$ and $\rho$ can be done in practice. Is an approach such as this meaningful at all in practice?

$\endgroup$
2
  • 3
    $\begingroup$ Not sure I understand (2), but as for (1): A confidence interval in general does not give you the probability that $\mu \in [a,b]$. See the excellent answer here. $\endgroup$ Commented May 27, 2021 at 11:48
  • $\begingroup$ Thanks for the answer, I was roughly aware of the subtlety arising from confidence interval interpretation, but that's certainly an answer worth reading. My question is roughly about how to use the convergence rate of CLT to account for the difference of the (rescaled) true distribution of the sample mean from the standard normal distribution. $\endgroup$
    – Uzu Lim
    Commented May 28, 2021 at 11:32

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.