2
$\begingroup$

For two sequence of random variables $X_1,\dots, X_n\sim_{iid} Beroulli(p) X_i$ and $Y_1,\dots, Y_n\sim_{iid} Beroulli(q)$ ($X_i$, $Y_j$ are independent), we have CLT $$ \sqrt{n}(\bar{X}_n-p)\to N(0,p(1-p)) $$ and $$ \sqrt{n}(\bar{Y}_n-q)\to N(0,q(1-q)) $$ Using Delta method, we get $$ \sqrt{n}(\log(\frac{\bar{X}_n}{1-\bar{X}_n})-\log (\frac{p}{1-p}))\to N(0, \frac{1}{p(1-p)}) $$

(1) Question: What is the distribution of $\sqrt{n}(\log(\frac{\bar{X}_n}{1-\bar{X}_n})-\log (\frac{p}{1-p})-\log(\frac{\bar{Y}_n}{1-\bar{Y}_n})-\log \frac{q}{1-q}))$?

Do we have $$\sqrt{n}(\log(\frac{\bar{X}_n}{1-\bar{X}_n})-\log (\frac{p}{1-p})-\log(\frac{\bar{Y}_n}{1-\bar{Y}_n})-\log \frac{q}{1-q}))\to N(0, \frac{1}{p(1-p)}-\frac{1}{q(1-q)})?$$ Otherwise, how to get it?

I think I am right. (2) Question: what is the consistent estimator of $\frac{1}{p(1-p)}-\frac{1}{q(1-q)}$?

$\endgroup$
8
  • $\begingroup$ Suppose $q=1/2:$ have you noticed your suggested variance of $p(1-p)-q(1-q)$ is negative? $\endgroup$
    – whuber
    Commented Feb 3, 2022 at 18:16
  • $\begingroup$ @whuber I edited my question. Thanks! $\endgroup$
    – Hermi
    Commented Feb 3, 2022 at 18:22
  • 1
    $\begingroup$ Now suppose $p=1/2:$ your suggested variance of $1/(p(1-p))-1/(q(1-q))$ again is negative. It sounds like you ought to begin with stats.stackexchange.com/questions/26886. $\endgroup$
    – whuber
    Commented Feb 3, 2022 at 18:37
  • 2
    $\begingroup$ The title asks for a different distribution than the text. Which one is the problem that you wish to get answered in this question?$$\sqrt{n}(\bar{X}_n-p)-\sqrt{n}(\bar{Y}_n-q)$$ versus $$\sqrt{n}(\log(\frac{\bar{X}_n}{1-\bar{X}_n})-\log (\frac{p}{1-p})-\log(\frac{\bar{Y}_n}{1-\bar{Y}_n})-\log \frac{q}{1-q}))$$ $\endgroup$ Commented Feb 3, 2022 at 19:25
  • 1
    $\begingroup$ What if $\bar{X}_n = 1$? Then the division $\frac{\bar{X}_n}{1-\bar{X}_n}$ is undefined (division by zero). Why do you make this transformation? $\endgroup$ Commented Feb 3, 2022 at 19:35

2 Answers 2

4
$\begingroup$

You're almost correct. As a reminder, you have to assume that $p, q \neq 0$.

Assume two independent sequences of random variables, $A_n$ and $B_n$, converge in distribution to $A$ and $B$: $A_n \rightarrow A$ and $B_n \rightarrow B$. Then for any real numbers, $c$ and $d$, we have $c \cdot A_n + d \cdot B_n \rightarrow c \cdot A + d \cdot B$.

In your example, we have $$A_n := \sqrt{n}\{\log(\frac{\bar{X}_n}{1-\bar{X}_n})-\log (\frac{p}{1-p})\}$$ $$B_n := \sqrt{n}\{\log(\frac{\bar{Y}_n}{1-\bar{Y}_n})-\log (\frac{q}{1-q})\},$$ $$A_n - B_n \rightarrow N\{0, \frac{1}{p(1-p)}\} - N\{0, \frac{1}{q(1-q)}\}$$.

You made a mistake in calculating your variance. For independent random variables $A$ and $B$, $var(A - B) = var(A) + var(B)$. So your two sequences converge to $$N\{0, \frac{1}{p(1-p)} + \frac{1}{q(1-q)}\}.$$

I see two ways to get a consistent estimate of the variance, $\frac{1}{p(1-p)} + \frac{1}{q(1-q)}$.

1. Multivariate Delta Method

Since we're assuming $\bar{X}_n$ and $\bar{Y}_n$ are $\sqrt{n}$ consistent estimators of $p$ and $q$, we know that

$$\sqrt{n} \begin{bmatrix} \bar{X}_n - p \\ \bar{Y}_n - q\ \end{bmatrix} \rightarrow N(0, \Sigma), \text{where}$$ $$\Sigma = \begin{bmatrix} p(1-p) & 0 \\ 0 & q(1-q) \\ \end{bmatrix}. $$

You can definitely estimate $\Sigma$ since it's the variance of two independent Binomial random variables. Plug $\hat{p}$ and $\hat{q}$ into $\Sigma$ to get $\hat{\Sigma}$.

Let $h(p ,q) = log(\frac{p}{1-p}) - log(\frac{p}{1-p})$ and let $\nabla h(p, q)$ be the gradient of $h$.

The Delta Method gives the asymptotic distribution and covariance matrix: $$\sqrt{n} \{h(\bar{X}_n, \bar{Y}_n) - h(p, q)\} \rightarrow N \{0, \nabla h(p, q)^T \cdot \Sigma \cdot \nabla h(p, q) \}$$

A consistent estimate of the variance is given by plugging $\hat{\Sigma}$ into the expression for the variance above.

2. Invariance Property of the Maximum Likelihood Estimate (MLE)

The Invariance Property of the MLE states that $\hat{\theta}$ is a MLE of $\theta$, then $f(\hat{\theta})$ is a MLE of $f(\theta)$. So plug $\hat{p} = \frac{1}{n} \sum_i X_i$ and $\hat{q} = \frac{1}{n} \sum_i Y_i$ into $f(p, q) = \frac{1}{p(1-p)} + \frac{1}{q(1-q)}$

$\endgroup$
4
  • $\begingroup$ Thanks! For the consistent estimate part, can I say that since $\bar{X}_n\to p$ and $\bar{Y}_n\to q$ in probability? This implies $\bar{X}_n$ and $\bar{Y}_n$ are consistent estimator. Then plug it into $f(p,q)$ to get the consistent estimator? $\endgroup$
    – Hermi
    Commented Feb 3, 2022 at 23:21
  • $\begingroup$ Yes to your first question, but I wasn’t thinking of it that way though. The sample mean is maximum likelihood estimate of a $p$ in a $Binom(n, p)$ so you know it’s consistent. $\endgroup$
    – Eli
    Commented Feb 4, 2022 at 4:08
  • $\begingroup$ @Bob: Try to solve it the same way, & post a new question if you should get stuck at some point. $\endgroup$ Commented Feb 4, 2022 at 8:44
  • $\begingroup$ @Bob: Please do not ask new questions in a comment, but as a new Question! $\endgroup$ Commented Feb 4, 2022 at 13:19
3
$\begingroup$

I'm answering the title question; the question in the body is unclear to me.


Using the rules for a variance operator we have:

$$\begin{align} \mathbb{V}(\sqrt{n} (\bar{X}_n-p) - \sqrt{n} (\bar{Y}_n-q)) &= \mathbb{V}(\sqrt{n} (\bar{X}_n-p)) + \mathbb{V}(\sqrt{n} (\bar{Y}_n-q)) \\[14pt] &= n \cdot \mathbb{V}(\bar{X}_n-p) + n \cdot \mathbb{V}(\bar{Y}_n-q) \\[14pt] &= n \cdot \mathbb{V}(\bar{X}_n) + n \cdot \mathbb{V}(\bar{Y}_n) \\[8pt] &= n \cdot \frac{p(1-p)}{n} + n \cdot \frac{q(1-q)}{n} \\[6pt] &= p(1-p) + q(1-q). \\[6pt] \end{align}$$

Since linear functions of normal random variables are normal random variables, you have the limiting distribution:

$$\sqrt{n} (\bar{X}_n-p) - \sqrt{n} (\bar{Y}_n-q) \overset{\text{approx}}{\sim} \text{N}(0, p(1-p) + q(1-q)).$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.