3
$\begingroup$

The Question
Consider a binary random variable X that satisfies: $Pr(X = 0) = \theta \ \ \ $ and $Pr(X = 1) = 1−\theta $ for $\theta \in (0, 1)$ an unknown parameter. Suppose an i.i.d. sample of size $n$ drawn from the distribution of X, $\{x_{i}, i = 1, \cdot \cdot \cdot , n\}$, is available, and $\hat{\theta}_{n} = \frac{1}{n}\sum^{n}_{i=1}x_{i}$ is considered as an estimator of θ.

Show that $\sqrt{n}(\hat{\theta}_{n} - \theta) \overset{d}{\rightarrow} N(0,\theta(1-\theta))\ as\ n \to \infty$

So this question is frustrating me, as it's a fundamental question, and I've really missed the big picture on it. I'll list two of the failed directions I look off in, but I'm just looking for some redirection here onto the right path.


Attempt #1
So for this approach, I just thought I'd brute force it. But I feel like this question is alluding to the CLT, so I was ready to abandon if it didn't feel like it was going to pay off....effectively my algebraic manipulation was: $$ \begin{align*} E[\sqrt{n}(\hat{\theta}_{n} - \theta)] &= \sqrt{n}E[(\frac{1}{n} \sum^{n}_{i=1} x_{i}) - \theta] \\ &= \frac{1}{\sqrt{n}}E[ \sum^{n}_{i=1} x_{i}] - \sqrt{n}E[\theta] \\ &= \frac{1}{\sqrt{n}} \cdot np - \sqrt{n}E[1-p]\\ &= \sqrt{n} p - \sqrt{n}(1-p)\\ &= \sqrt{n}(2p-1) \end{align*} $$

Which is not great, as this will explode as $\lim_{n\to\infty}$


Attempt #2
I was very much following the proof of the CLT that uses characteristic functions. i.e. starting with: $$ \begin{align*} Z_{n} &= \frac{n(1/n)\sum x_{i} - n\theta}{n(\sigma / \sqrt{n})} \\ ...\\ &= \sum \left( \frac{Y_{i}}{\sqrt{N}} \right), \ \ Y_{i}=\frac{x_{i}-\theta}{\sigma} \end{align*} $$

But inevitably, this leads me to expanding out $\varphi_{Z_{n}}(t) = \prod_{i=1}^{N}\varphi_{Y}\left(\frac{t}{\sqrt{n}}\right)$ and showing that my standardized variable is $\sim N(0,1)$, which is a nice regurgitation of the proof, but a failure on my part to adapt it.


I'm keen to know, were any of my approaches getting close, and what fundamental have I failed to realize?

$\endgroup$
2
  • $\begingroup$ In attempt #1 what do you think $E(X_i)$ is? Showing what the expectation is won't tell you the distribution, of course; you'd have to establish that. $\endgroup$
    – Glen_b
    Commented Apr 20, 2015 at 22:29
  • $\begingroup$ Are you supposed to prove it without using the CLT? I.e. without taking the CLT as a known result. $\endgroup$
    – KOE
    Commented Apr 20, 2015 at 22:38

1 Answer 1

2
$\begingroup$

There's a bug in your question definition: If $\theta = \Pr\{X=0\}$, then the estimator for $\theta$ should be $\hat\theta = 1-\frac{1}{n}\sum x_i$ and not what you wrote.

Here's why: suppose that $\theta = \Pr\{X=0\}$ is large (=close to 1). Then, $X$ will be mostly zero, so $\frac{1}{n}\sum x_i$ will be close to zero!

It is more common to define $\theta = \Pr\{X_i = 1\}$, and with this notation, $\hat\theta$ will be indeed $\hat\theta = \frac{1}{n}\sum x_i$.

This will fix the problem with Attempt #1 (you will get zero at the end). Also, try not to use both $p$ and $\theta$ do describe the same thing...

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.