Slutsky's theorem applied to Bernoulli random variables

Question

Suppose $X_n$ is a sequence of $n$ Bernoulli random variables with unknown $p$, and I try to get a confidence interval for $p$. Using central limit theorem, I've got

\begin{align*} \frac{\bar X_n - p}{\sqrt{(p(1-p)/n}} \sim \mathcal{N}(0, 1) \\ \end{align*}

Suppose

$$\mathbb{P}\Bigg(\left| \frac{\bar X_n - p}{\sqrt{(p(1-p)/n}} \right|\Bigg) \le t $$

then we get a confidence interval for $p$ with a probability of $t$:

$$ \mathcal{I}_{t} = \Big[\bar X_n - \frac{t\sqrt{p(1-p)}}{\sqrt{n}}, \bar X_n + \frac{t\sqrt{p(1-p)}}{\sqrt{n}} \Big] $$

Unfortunately, this interval depends on $p$. I've learned that somehow we could replace $p$ with $\bar X_n$ when $n \to \infty$, and the interval becomes

$$ \mathcal{I}_t' = \Big[\bar X_n - \frac{t\sqrt{\bar X_n(1-\bar X_n)}}{\sqrt{n}}, \bar X_n + \frac{t\sqrt{\bar X_n(1-\bar X_n)}}{\sqrt{n}} \Big] $$

I feel it may have something related to Slutsky's theorem, but I don't how it could be applied in this case. Could anybody explain with some more details, please?

Answer 1

Slutsky's theorem says that if $X_n \xrightarrow[n \to \infty]{(d)} X$ and $Y_n \xrightarrow[n \to \infty]{\mathbb{P}} c$, then

\begin{align*} X_n Y_n &\xrightarrow[n \to \infty]{(d)} cX \\ X_n + Y_n &\xrightarrow[n \to \infty]{(d)} X + c \end{align*}

• $\xrightarrow[n \to \infty]{(d)}$ means convergence in distribution
• $\xrightarrow[n \to \infty]{\mathbb{P}}$ means convergence in probability

NOTE: if $X$ in the above example is also a constant, the result is strengthened to convergence in probability.

In your example we have $\bar{X} \xrightarrow[n \to \infty]{\mathbb{P}} p$ by the Weak Law of Large Numbers. Hence, if you can show that

$$\sqrt{\frac{\bar{X_n}(1-\bar{X_n})}{n}} \xrightarrow[n \to \infty]{(d)} \sqrt{\frac{p(1-p)}{n}}$$

then by applying Slutsky's theorem will tell you that their sum converges in probability:

$$\bar{X_n} \pm t\sqrt{\frac{\bar{X_n}(1-\bar{X_n})}{n}} \xrightarrow[n \to \infty]{\mathbb{P}} \bar{X_n} \pm t\sqrt{\frac{p(1-p)}{n}}$$

To prove $\sqrt{\frac{\bar{X_n}(1-\bar{X_n})}{n}} \xrightarrow[n \to \infty]{(d)} \sqrt{\frac{p(1-p)}{n}}$, note that by the central limit theorem we have

$$\sqrt{n}(\bar{X_n} - p) \xrightarrow[n \to \infty]{(d)} N(0,p(1-p))$$

Since $\bar{X_n}$ is the average of i.i.d $\rm{Bernoulli}(p)$ random variables. Let $f(x) = \sqrt{x(1-x)}$ which is continuous on the support of $X_n$ (since $X_n \geq 0)$. So we can apply the Delta method to obtain

$$\sqrt{n}\left(\sqrt{\bar{X_n}(1-\bar{X_n})} - \sqrt{p(1-p)}\right) \xrightarrow[n \to \infty]{(d)} N(0,f'(p)^2 p(1-p))$$

That is,

$$\sqrt{\bar{X_n}(1-\bar{X_n})} \xrightarrow[n \to \infty]{(d)} N\left(\sqrt{p(1-p)}, \frac{f'(p)^2 p(1-p)}{n}\right)$$

Notice the variance's numerator only depends on $p$, so it's fixed, but the denominator depends on $n$. So as $n\to \infty$ the variance goes to $0$, and hence

$$\sqrt{\bar{X_n}(1-\bar{X_n})} \xrightarrow[n \to \infty]{\mathbb{P}} \sqrt{p(1-p)}$$

It immediately follows that

$$\sqrt{\frac{\bar{X_n}(1-\bar{X_n})}{n}} \xrightarrow[n \to \infty]{\mathbb{P}} \sqrt{\frac{p(1-p)}{n}}$$

and hence distribution, as required.