16
$\begingroup$

Let's assume that we have samples of two independent Bernoulli random variables, $\mathrm{Ber}(\theta_1)$ and $\mathrm{Ber}(\theta_2)$.

How do we prove that $$\frac{(\bar X_1-\bar X_2)-(\theta_1-\theta_2)}{\sqrt{\frac{\theta_1(1-\theta_1)}{n_1}+\frac{\theta_2(1-\theta_2)}{n_2}}}\xrightarrow{d} \mathcal N(0,1)$$?

Assume that $n_1\neq n_2$.

Improve this question
$\endgroup$
16
  • $\begingroup$ Z_i= X_1i - X_2i is a sequence of iid rv of finite mean and variance. Hence it satisfies the Levy-Linderberg central limit theorem from which your results follow. Or are you asking for a proof of the clt itself? $\endgroup$
    – Three Diag
    Feb 17, 2017 at 15:48
  • $\begingroup$ @ThreeDiag How are you applying the LL version of the CLT? I don't think that's correct. Write an answer for me to check the details. $\endgroup$
    – An old man in the sea.
    Feb 17, 2017 at 20:13
  • $\begingroup$ All the details are already there. For LL to apply you need a sequence of iid rv with finite mean and variance. The variable Z_i = X_i1 and X_i2 satisfies all three requirements. Independence follows from independence of the two original bernoulli vars and you can see that E(Z_i) and V(Z_i) are finite by applying standard properties of E and V $\endgroup$
    – Three Diag
    Feb 17, 2017 at 22:58
  • 1
    $\begingroup$ "samples of two independent Bernoulli random variables" - incorrect expression. Must be: "two independent samples from Bernoulli distributions". $\endgroup$
    – Viktor
    Dec 14, 2017 at 11:05
  • 1
    $\begingroup$ Please add "as $n_1,n_2\to \infty$". $\endgroup$
    – Viktor
    Dec 14, 2017 at 13:45

2 Answers 2

Reset to default
10
+100
$\begingroup$

Put $a=\frac{\sqrt{\theta_1(1-\theta_1)}}{\sqrt{n_1}}$, $b=\frac{\sqrt{\theta_2(1-\theta_2)}}{\sqrt{n_2}}$, $A=(\bar{X}_1-\theta_1)/a$, $B=(\bar{X}_2-\theta_2)/b$. We have $A\to_d N(0,1),\ B\to_d N(0,1)$. In terms of characteristic functions it means $$ \phi_A(t)\equiv {\bf E}e^{itA}\to e^{-t^2/2},\ \phi_B(t)\to e^{-t^2/2}.$$ We want to prove that $$D:=\frac{a}{\sqrt{a^2+b^2}}A-\frac{b}{\sqrt{a^2+b^2}}B\to_d N(0,1) $$

Since $A$ and $B$ are independent, $$\phi_D(t)=\phi_A\left(\frac{a}{\sqrt{a^2+b^2}}t\right)\phi_B\left(-\frac{b}{\sqrt{a^2+b^2}}t\right)\to e^{-t^2/2}, $$ as we wish it to be.

This proof is incomplete. Here we need some estimates for uniform convergence of characteristic functions. However in the case under consideration we can do explicit calculations. Put $p=\theta_1,\ m=n_1$. \begin{align} \phi_{X_{1,1}}(t) &= 1+p(e^{it}-1), \\ \phi_{\bar X_{1}}(t) &= (1+p(e^{it/m}-1))^m, \\ \phi_{\bar X_{1}-\theta_1}(t) &= (1+p(e^{it/m}-1))^m e^{-ipt}, \\ \phi_{A}(t) &= (1+p(e^{it/\sqrt{mp(1-p)}}-1))^m e^{-ipt\sqrt{m}/\sqrt{p(1-p)}} \\[5pt] &= \left( \left(1+p(e^{it/\sqrt{mp(1-p)}}-1)\right)e^{-ipt/\sqrt{mp(1-p)}}\right)^m \\[5pt] &=\left( 1-\frac{t^2}{2m}+O(t^3m^{-3/2}) \right)^m \end{align} as $t^3m^{-3/2}\to 0$. Thus, for a fixed $t$, $$\phi_D(t)=\left( 1-\frac{a^2t^2}{2(a^2+b^2)n_1}+O(n_1^{-3/2}) \right)^{n_1} \left( 1-\frac{b^2t^2}{2(a^2+b^2)n_2}+O(n_2^{-3/2}) \right)^{n_2} \to e^{-t^2/2} $$ (even if $a\to 0$ or $b\to 0$), since $\left|e^{-y}-(1-y/m)^m\right|\le {y^2}/{2m}\ $ when $\ y/m<1/2$ (see https://math.stackexchange.com/questions/2566469/uniform-bounds-for-1-y-nn-exp-y/ ).

Note that similar calculations may be done for arbitrary (not necessarily Bernoulli) distributions with finite second moments, using the expansion of characteristic function in terms of the first two moments.

Improve this answer
$\endgroup$
1
  • $\begingroup$ This seems correct. I'll get back to you later on, when I have time to check everything. ;) $\endgroup$
    – An old man in the sea.
    Dec 14, 2017 at 11:10
-1
$\begingroup$

Proving your statement is equivalent to proving the (Levy-Lindenberg) Central Limit Theorem which states

If $\{Z_i\}_{i=1}^n$ is a sequence of i.i.d random variable with finite mean $\mathbb{E}(Z_i) = \mu $ and finite variance $\mathbb{V}(Z_i) = \sigma^2$ then $$ \sqrt{n}(\bar{Z} - \mu) \to^d N(0,\sigma^2)$$

Here $\bar{Z} = \sum_i Z_i/n$ that is the sample variance.

Then it is easy to see that if we put

$$ Z_i = X_1i - X_2i $$ with $X_{1i}, X_{2i}$ following a $Ber(\theta_1)$ and $Ber(\theta_2)$ respectively the conditions for the theorem are satisfied, in particular

$$ \mathbb{E}(Z_i) = \theta_1 - \theta_2 = \mu $$

and

$$ \mathbb{V}(Z_i)= \theta_1(1-\theta_1) +\theta_2(1-\theta_2)= \sigma^2 $$

(There's a last passage, and you have to adjust this a bit for the general case where $n_1 \neq n_2$ but I have to go now, will finish tomorrow or you can edit the question with the final passage as an exercise )

Improve this answer
$\endgroup$
6
  • $\begingroup$ I could not obtain what I wanted exactly because of the possibility of $n_1\neq n_2$ $\endgroup$
    – An old man in the sea.
    Feb 18, 2017 at 19:29
  • $\begingroup$ I will show later if you can't get it. Hint: compute the variance of the sample mean of Z and use that as the variable in the theorem $\endgroup$
    – Three Diag
    Feb 18, 2017 at 19:33
  • $\begingroup$ Three, could you please add the details for when $n_1 \neq n_2$? Thanks $\endgroup$
    – An old man in the sea.
    Feb 21, 2017 at 9:27
  • $\begingroup$ Will do as soon as find a little timr. There was in fact a subtlety that prevents from using LL clt without adjustment. There are three ways to go, the simplest of which is invoking the fact that for large n1 and n2, X1 and X2 go in distribution to normals, then a linear combination of normal is also normal. This is a property of normals that you can take as given, otherwise you can prove it by characteristic functions. $\endgroup$
    – Three Diag
    Feb 22, 2017 at 8:21
  • $\begingroup$ The other two require either a different clt (Lyapunov possibly) or alternatively treat n1 = i and n2= i +k. Then for large i you can essentially disregard k and you can go back to apply LL (but still it will require some care to nail the right variance) $\endgroup$
    – Three Diag
    Feb 22, 2017 at 8:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.