Suppose $X_n$ is a sequence of $n$ Bernoulli random variables with unknown $p$, and I try to get a confidence interval for $p$. Using central limit theorem, I've got

\begin{align*} \frac{\bar X_n - p}{\sqrt{(p(1-p)/n}} \sim \mathcal{N}(0, 1) \\ \end{align*}


$$\mathbb{P}\Bigg(\left| \frac{\bar X_n - p}{\sqrt{(p(1-p)/n}} \right|\Bigg) \le t $$

then we get a confidence interval for $p$ with a probability of $t$:

$$ \mathcal{I}_{t} = \Big[\bar X_n - \frac{t\sqrt{p(1-p)}}{\sqrt{n}}, \bar X_n + \frac{t\sqrt{p(1-p)}}{\sqrt{n}} \Big] $$

Unfortunately, this interval depends on $p$. I've learned that somehow we could replace $p$ with $\bar X_n$ when $n \to \infty$, and the interval becomes

$$ \mathcal{I}_t' = \Big[\bar X_n - \frac{t\sqrt{\bar X_n(1-\bar X_n)}}{\sqrt{n}}, \bar X_n + \frac{t\sqrt{\bar X_n(1-\bar X_n)}}{\sqrt{n}} \Big] $$

I feel it may have something related to Slutsky's theorem, but I don't how it could be applied in this case. Could anybody explain with some more details, please?


Slutsky's theorem says that if $X_n \xrightarrow[n \to \infty]{(d)} X$ and $Y_n \xrightarrow[n \to \infty]{\mathbb{P}} c$, then

\begin{align*} X_n Y_n &\xrightarrow[n \to \infty]{(d)} cX \\ X_n + Y_n &\xrightarrow[n \to \infty]{(d)} X + c \end{align*}

  • $\xrightarrow[n \to \infty]{(d)}$ means convergence in distribution
  • $\xrightarrow[n \to \infty]{\mathbb{P}}$ means convergence in probability

NOTE: if $X$ in the above example is also a constant, the result is strengthened to convergence in probability.

In your example we have $\bar{X} \xrightarrow[n \to \infty]{\mathbb{P}} p$ by the Weak Law of Large Numbers. Hence, if you can show that

$$\sqrt{\frac{\bar{X_n}(1-\bar{X_n})}{n}} \xrightarrow[n \to \infty]{(d)} \sqrt{\frac{p(1-p)}{n}}$$

then by applying Slutsky's theorem will tell you that their sum converges in probability:

$$\bar{X_n} \pm t\sqrt{\frac{\bar{X_n}(1-\bar{X_n})}{n}} \xrightarrow[n \to \infty]{\mathbb{P}} \bar{X_n} \pm t\sqrt{\frac{p(1-p)}{n}}$$

To prove $\sqrt{\frac{\bar{X_n}(1-\bar{X_n})}{n}} \xrightarrow[n \to \infty]{(d)} \sqrt{\frac{p(1-p)}{n}}$, note that by the central limit theorem we have

$$\sqrt{n}(\bar{X_n} - p) \xrightarrow[n \to \infty]{(d)} N(0,p(1-p))$$

Since $\bar{X_n}$ is the average of i.i.d $\rm{Bernoulli}(p)$ random variables. Let $f(x) = \sqrt{x(1-x)}$ which is continuous on the support of $X_n$ (since $X_n \geq 0)$. So we can apply the Delta method to obtain

$$\sqrt{n}\left(\sqrt{\bar{X_n}(1-\bar{X_n})} - \sqrt{p(1-p)}\right) \xrightarrow[n \to \infty]{(d)} N(0,f'(p)^2 p(1-p))$$

That is,

$$\sqrt{\bar{X_n}(1-\bar{X_n})} \xrightarrow[n \to \infty]{(d)} N\left(\sqrt{p(1-p)}, \frac{f'(p)^2 p(1-p)}{n}\right)$$

Notice the variance's numerator only depends on $p$, so it's fixed, but the denominator depends on $n$. So as $n\to \infty$ the variance goes to $0$, and hence

$$\sqrt{\bar{X_n}(1-\bar{X_n})} \xrightarrow[n \to \infty]{\mathbb{P}} \sqrt{p(1-p)}$$

It immediately follows that

$$\sqrt{\frac{\bar{X_n}(1-\bar{X_n})}{n}} \xrightarrow[n \to \infty]{\mathbb{P}} \sqrt{\frac{p(1-p)}{n}}$$

and hence distribution, as required.

  • $\begingroup$ That's insightful! But how to show $\sqrt{\frac{\bar{X_n}(1-\bar{X_n})}{n}} \to \sqrt{\frac{p(1-p)}{n}}$? I almost took it for granted, as the convergence in the numerator obvious, but the denominator also goes to $\infty$. Is there a rigorous way to prove it, please? $\endgroup$ – zyxue Oct 4 '18 at 4:21
  • 1
    $\begingroup$ Added the proof at the end of my comment. This stuff becomes easy once you've done a few proofs using CLT/Slutsky's theorem/Delta method $\endgroup$ – Xiaomi Oct 4 '18 at 4:41
  • $\begingroup$ Is it true that if $\bar X_n \to p$, then $\sqrt{\bar X_n} \to \sqrt{p}$? $\endgroup$ – zyxue Oct 8 '18 at 20:42
  • 1
    $\begingroup$ Yes, by the continuous mapping theorem since $\bar{X_n}$ is non-negative. $\endgroup$ – Xiaomi Oct 8 '18 at 23:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.