2
$\begingroup$

I'm reading introduction to mathematical statistics by R. Hogg, et al. I have some trouble to understand a limiting distribution.

Let $X_1,\cdots,X_{n_1}$ be iid random variables from $Bernoulli(p_1)$ and let $Y_1,\cdots,Y_{n_2}$ be iid random variables from $Bernoulli(p_2)$. The hypothesis of interest are given as follows: $$H_0: p_1=p_2\ \text{vs}.\ H_1: p_1\ne p_2.$$

From the Central Limit Theorem, I think we can have $$\frac{\sqrt{n_i}(\hat{p}_i-p_i)}{\sqrt{p_i(1-p_i)}}\to Z_i\ \text{in distribution},\ i=1,2,$$ since $\hat{p_1}=(\sum_{i=1}^{n_1} x_i)/n_1$, then it is the sum of independent random variables (of course it has $1/n$).

Assume for $i=1,2$, as $n\to\infty,\ n_i/n\to\lambda_i$, where $0<\lambda_i<1$ and $\lambda_1+\lambda_2=1$. Then I'd like to prove that $$\sqrt{n}[(\hat{p_1}-\hat{p_2})-(p_1-p_2)]\to N(0,\frac{1}{\lambda_1}p_1(1-p_1)+\frac{1}{\lambda_2}p_2(1-p_2)).$$

What I've tried
I think I can't just add two limiting distributions from the CLT since the addition doesn't hold for convergence in distribution.
I also tried to prove it with CLT, but I think I can't use it since $\hat{p_1}-\hat{p_2}$ is not the sum of iid variables although $\hat{p_1}-\hat{p_2}=(\sum X_i)/{n_1} - (\sum Y_j)/{n_2}$ is independent.
Can you explain how to prove the convergence in distribution?

$\endgroup$
1
  • $\begingroup$ If $n_1$ and $n_2$ add to give $n$, $n_1/n$ and $n_2/n$ are not independent. $\endgroup$
    – Glen_b
    Commented May 18 at 1:32

1 Answer 1

2
$\begingroup$

Given that two samples $\{X_i\}$ and $\{Y_j\}$ are also independent (you implied this condition in your title and attempt but didn't make it explicit in the problem statement), \begin{align*} & \sqrt{n_1}(\hat{p}_1 - p_1) \to_d Z_1 \sim N(0, p_1(1 - p_1)), \text{ as } n_1 \to \infty, \tag{1}\label{1} \\ & \sqrt{n_2}(\hat{p}_2 - p_2) \to_d Z_2 \sim N(0, p_2(1 - p_2)), \text{ as } n_2 \to \infty, \tag{2}\label{2} \end{align*} as well as $\frac{n_i}{n} \to \lambda_i \in (0, 1)$ as $n \to \infty$, intuitively, it follows that \begin{align*} & \sqrt{n}[(\hat{p}_1 - \hat{p}_2) - (p_1 - p_2)] \\ =& \sqrt{n}(\hat{p}_1 - p_1) - \sqrt{n}(\hat{p}_2 - p_2) \\ =& \sqrt{\frac{n}{n_1}}\sqrt{n_1}(\hat{p}_1 - p_1) - \sqrt{\frac{n}{n_2}}\sqrt{n_2}(\hat{p}_2 - p_2) \\ \to_d & \sqrt{\lambda_1^{-1}}Z_1 - \sqrt{\lambda_2^{-1}}Z_2 \sim N(0, \lambda_1^{-1}p_1(1 - p_1) + \lambda_2^{-1}p_2(1 - p_2)) \tag{3}\label{3} \end{align*} as $\color{red}{n \to \infty}$. Hence the conclusion holds. Clearly, the last "$\to$" step needs additional justifications.

First, in order to apply $\eqref{1}$ and $\eqref{2}$ as $\color{red}{n \to \infty}$ , we need to ensure that $n_i \to \infty$ as $n \to \infty$, which follows by writing $n_i = \frac{n_i}{n} \times n$ and the condition $n_i/n \to \lambda_i \color{red}{>} 0$.

Then, the Slutsky's theorem guarantees that for $i = 1, 2$: \begin{align*} \sqrt{\frac{n}{n_i}}\sqrt{n_i}(\hat{p}_1 - p_1) \to_d \sqrt{\lambda_i^{-1}}Z_i \tag{4}\label{4} \end{align*} as $n \to \infty$.

Now if $Z_1$ and $Z_2$ are independent, then the "$\sim$" relation in $\eqref{3}$ holds, and the proof is complete. However, it seems quite difficult to give it a rigorous proof. This motivates me to consider exploiting the independence between $\{X_i\}$ and $\{Y_j\}$ from a different angle.

By Lévy's continuity theorem, to show $\eqref{3}$, it is equivalent to show the characteristic function of the random variable $\sqrt{n}(\hat{p}_1 - p_1) - \sqrt{n}(\hat{p}_2 - p_2)$ converges point-wisely $e^{-\frac{1}{2}\left(\frac{p_1(1 - p_1)}{\lambda_1} + \frac{p_2(1 - p_2)}{\lambda_2}\right)t^2}$, which is the characteristic function of a $N(0, \lambda_1^{-1}p_1(1 - p_1) + \lambda_2^{-1}p_2(1 - p_2))$ random variable. To this end, we have \begin{align*} & E\left[e^{it\sqrt{n}((\hat{p}_1 - p_1) - (\hat{p}_2 - p_2))}\right] \\ =& E\left[e^{it\sqrt{n}(\hat{p}_1 - p_1)}\right]E\left[e^{i(-t)\sqrt{n}(\hat{p}_2 - p_2)}\right] \tag{5.1}\label{5.1} \\ =& E\left[e^{it\sqrt{\frac{n}{n_1}}\sqrt{n_1}(\hat{p}_1 - p_1)}\right]E\left[e^{i(-t)\sqrt{\frac{n}{n_2}}\sqrt{n_2}(\hat{p}_2 - p_2)}\right] \\ \to & e^{-\frac{1}{2}\lambda_1^{-1}p_1(1 - p_1)t^2}e^{-\frac{1}{2}\lambda_2^{-1}p_2(1 - p_2)(-t)^2} \tag{5.2}\label{5.2} \\ =& e^{-\frac{1}{2}\left(\frac{p_1(1 - p_1)}{\lambda_1} + \frac{p_2(1 - p_2)}{\lambda_2}\right)t^2} \end{align*} as $n \to \infty$. This completes the proof. In $\eqref{5.1}$, we used the condition that $\hat{p}_1$ and $\hat{p}_2$ are independent (which is a consequence of $\{X_i\}$ and $\{Y_j\}$ are independent). In $\eqref{5.2}$, we used the established result $\eqref{4}$ and Lévy's continuity theorem (the other direction).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.