I read from http://math.arizona.edu/~jwatkins/N_unbiased.pdf

We know the Fisher information is $$I(\theta)=E\bigg[\bigg(\frac{\partial \log f(X) }{\partial \theta}\bigg)^2\bigg]. $$

Cramer-Rao lower bound is $\frac{(h'(\theta))^2}{I(\theta)}$ where $h'(\theta)=E\bigg[d(X) \frac{\partial \log f(X) }{\partial \theta}\bigg]. $

Then in page 214, example 14.15 considered the independent Bernoulli random variables with unknown success probability $\theta$. So density function is $\theta^x (1-\theta)^{1-x}$. In this case $I(\theta)=\frac{1}{\theta\left(1-\theta\right)}$.

Now if we take $d(X)=\bar X,$ then $h'(\theta)=E\bigg[\bar X \frac{\partial \log f(X) }{\partial \theta}\bigg]$. Then how to calculate $h'(\theta)$?

It seems $E\bigg[\bar X \frac{\partial \log f(\bar X) }{\partial \theta}\bigg]= E(\bar X \frac{\bar X -\theta}{\theta (1-\theta)})= \frac{1}{\theta(1-\theta)} E( \bar X ^2 - \theta \bar X)= \frac{1}{\theta (1-\theta)}Var (\bar X)= \frac{1}{n}$.

We know $Var(\bar X)=\frac{\theta(1-\theta)}{n}$. Also $I(\theta)=\frac{1}{\theta (1-\theta)}$. If $h'(\theta)=1$ then $Var(\bar X) < \frac{(h'(\theta))^2}{I(\theta)},$ contradicts the CR bound.


1 Answer 1


Your question seems not clear to me. How you get the last equation$E\bigg[\bar X \frac{\partial \ log f(\bar X) }{\partial \theta}\bigg]=\frac{\theta(1-\theta)}{n}$ without knowing the distribution $X$?

From $(14.12)$

$$h(\theta)=E_\theta [d(X)]=\int_{R^n}d(x)f(x|\theta)dx$$ Then

\begin{align*} h'(\theta)=\int_{R^n}d(x)\frac{\partial f(x|\theta)}{\partial \theta}dx&=\int_{R^n}d(x)\frac{\partial f(x|\theta)}{\partial \theta}\frac{f(x|\theta)}{f(x|\theta)}dx\\&=\int_{R^n}d(x)\frac{\partial \log(f(x|\theta))}{\partial \theta}f(x|\theta)dx \tag{1} \end{align*}

i.e. $(1)$ is

$$h'(\theta)=E_\theta\left [d(x)\frac{\partial \log(f(x|\theta))}{\partial \theta}\right] \tag{14.14}$$

Therefore, if you want to calculate $h'(\theta)$ just calculate directly by $(1)$ but you need to know the distribution of $X$

Ok, since you update the question, and $X$ has a Bernoulli distrbituion and your $d(X)$ is the $\bar{X}$

Then why not directly calculte the $h'(\theta)$ by the definiton of $(14.12)$



  • $\begingroup$ Sorry it is coming $\frac{1}{n}$ $\endgroup$
    – str
    Nov 16, 2017 at 11:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.