Consider when $X_1, ..., X_n \sim $ Bernoulli($p$). We want to estimate $p(1-p)$. Suppose $\hat{p}=\frac{1}{n}\sum_{i=1}^nX_i$. Prove that the MLE $\hat{p}(1-\hat{p})$ is a asymptotically efficient

Here efficient estimator is referring to "any estimator that attain equality of the Cramér-Rao bound asymptotically". In other words for an estimator $T$ we have $ \operatorname{Var}[\,T\,]\ =\ \mathcal{I}_\theta^{-1}$.

Here is my confusion about the problem: We know that MLE estimators are (almost) always asymptotically efficient. Can we just use this fact?

One might also say that, find $v = Var(\hat{p}(1-\hat{p}))$ and using $i = I(p(1-p))$ (Fisher information) find the the efficiency $e=vi$. However I am not aware of straightforward closed forms for $i$ and $v$.

Another idea is to use Delta method and say, we know: $$ \sqrt{n} ( p - \hat{p} ) \rightarrow N(0, p(1-p)/n) $$ then, defining $g(u) = u(1-u)$ and $g'(u) = 1-2u$, and using Delta method we have: $$ \sqrt{n} ( g(p) - g(\hat{p}) ) \rightarrow N(0, (1-2p)^2{p(1-p)}/n) $$ So asymptotically $Var(\hat{p}(1-\hat{p})) = (1-2p)^2{p(1-p)}/n$. Also we know $I_X=nI_{X_1}$. $I_{X_1}( {p}(1-{p}) ) = \mathbb{E} \left[ -\frac{d^2}{dg({p})^2} \log f(X_1) \right] $. However this seems to hard to calculate.

Update: I calculate the fisher information for $p$: $ l(p; x) = \ln f(x|p) = x \ln p +(1-x) \ln 1-p \Rightarrow \\ \frac{d}{dp} l(p; x) = x / p -(1-x) / (1-p) \Rightarrow \\ \frac{d^2}{dp^2} l(p; x) = -x / p^2 -(1-x) / (1-p)^2 \Rightarrow \\ I_{X_1}(p) = \mathbb{E}_x[-l(p; x)] = p / p^2 +(1-p) / (1-p)^2 = \frac{1}{p} - \frac{1}{1-p} = \frac{1}{p(1-p)} \Rightarrow \\ I_{X}(p) = \frac{n}{p(1-p)} \\ $

Using the idea by @Xi'an (in the comments):

$ I_{X}(p) = I_X(g(p)) \left( 1-2p \right)^2 $ $$ \Rightarrow 1/I_X(g(p)) = \frac{p(1-p)(1-2p)^2}{n} $$ Which would prove the asymptotic efficiency of the estimator as $Var(g(\hat{p}))/I_X(g(p))=1$.

Caution: This is a HW problem. Hints are appreciated.

  • 1
    $\begingroup$ Homework problems should normally carry the self-study tag. See its tag wiki. $\endgroup$ – Glen_b Dec 15 '14 at 8:02
  • $\begingroup$ In respect of $v$, can you compute either $\text{Var}(\hat{p}^2)$ or $E(\hat{p}^4)$ or some other quantity which is a function of $E(\hat{p}^4)$ and lower order moments (like, say, the fourth factorial moment)? $\endgroup$ – Glen_b Dec 15 '14 at 8:35
  • $\begingroup$ I added the tag $\endgroup$ – Daniel Dec 15 '14 at 12:29
  • $\begingroup$ I can calculate both of $Var(\hat{p}^2)$ and $\mathbb{E}(\hat{p}^4)$ (using Delta method) but I don't know where you are going with this. $\endgroup$ – Daniel Dec 15 '14 at 12:34
  • $\begingroup$ a. Could you set the definition of asymptotically efficient as your starting point? It is usually helpful to rely on definitions. $\endgroup$ – Xi'an Dec 15 '14 at 14:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.