For $X_1,\dots, X_n$ iid sample from $X\sim Bernoulli(p)$. I try to verity that the estimator $\hat{p}=\bar{X}$ (sample mean) is the UMVUE for unknown parameter $p$.

I know that $\hat{p}$ is unbiased. I try to show that $Var[\hat{p}]=CRLB$ (Cramer-Rao lower bound), which is $$ CRLB=\frac{[I'(\theta)]^2}{nI(\theta)}, $$ and $I$ is the Fisher information.

My question is that what is the Fisher information here?

Note that the score function is $$U(p)=\frac{x-p}{p(1-p)}$$ for one sample.

Also, $$U(p)=\frac{\sum x_i-np}{p(1-p)}$$ for $n$ samples $X_1,\dots, X_n$.

I have two $I(p)$ here. I am not sure if I choose the Fisher information based on the $U(p)$ for one sample or $n$ sample...

$$ I_1=E\left[\frac{(X-p)^2}{(p(1-p))^2}\right]=\frac{1}{p(1-p)} $$ and another one is $$ I_2=E\left[\frac{(\sum X_i-np)^2}{(p(1-p))^2}\right]=\frac{n}{p(1-p)} $$


1 Answer 1


Cramér-Rao Lower Bound would be of the form $\operatorname{Var}_\theta(T(\mathbf X) ) \geq \mathscr I(\theta)^{-1}. $ For exponential family, $\mathscr I(\theta) =\mathbb E_\theta\left[-\partial^2_\theta \ln f(\mathbf x;\theta)\right],$ which for $X_i\overset{\text{i.i.d.}}{\sim}\mathrm{Ber}(p) $ is

\begin{align}\mathscr I(p) &=\mathbb E_p\left[-\partial^2_p\ln f(\mathbf x;p)\right]\\&=\mathbb E_p\left[-\sum_{i=1}^n\partial^2_p \ln f( x_i;p)\right]\\&=\mathbb E_p\left[-\sum_{i=1}^n\left\{-\frac{x_i}{p^2}+\frac{1-x_i}{(1-p)^2}\right\}\right]\\&=n[p(1-p)]^{-1}.\tag 1\label 1\end{align}

From $\eqref 1,$ it follows the CRLB is $\frac{p(1-p)}n.$

  • $\begingroup$ So use the $n$ data sample? Not just use the one data point? $\endgroup$
    – Hermi
    Mar 7 at 17:14
  • $\begingroup$ Yes @Hermi. You are working with a sample. You are inferring based on $n$ observations. It must get reflected in your calculation. $\endgroup$ Mar 7 at 17:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.