I read from http://math.arizona.edu/~jwatkins/N_unbiased.pdf

We know the Fisher information is $$I(\theta)=E\bigg[\bigg(\frac{\partial \log f(X) }{\partial \theta}\bigg)^2\bigg]. $$

Cramer-Rao lower bound is $\frac{(h'(\theta))^2}{I(\theta)}$ where $h'(\theta)=E\bigg[d(X) \frac{\partial \log f(X) }{\partial \theta}\bigg]. $

Then in page 214, example 14.15 considered the independent Bernoulli random variables with unknown success probability $\theta$. So density function is $\theta^x (1-\theta)^{1-x}$. In this case $I(\theta)=\frac{1}{\theta\left(1-\theta\right)}$.

Now if we take $d(X)=\bar X,$ then $h'(\theta)=E\bigg[\bar X \frac{\partial \log f(X) }{\partial \theta}\bigg]$. Then how to calculate $h'(\theta)$?

It seems $E\bigg[\bar X \frac{\partial \log f(\bar X) }{\partial \theta}\bigg]= E(\bar X \frac{\bar X -\theta}{\theta (1-\theta)})= \frac{1}{\theta(1-\theta)} E( \bar X ^2 - \theta \bar X)= \frac{1}{\theta (1-\theta)}Var (\bar X)= \frac{1}{n}$.

We know $Var(\bar X)=\frac{\theta(1-\theta)}{n}$. Also $I(\theta)=\frac{1}{\theta (1-\theta)}$. If $h'(\theta)=1$ then $Var(\bar X) < \frac{(h'(\theta))^2}{I(\theta)},$ contradicts the CR bound.


Your question seems not clear to me. How you get the last equation$E\bigg[\bar X \frac{\partial \ log f(\bar X) }{\partial \theta}\bigg]=\frac{\theta(1-\theta)}{n}$ without knowing the distribution $X$?

From $(14.12)$

$$h(\theta)=E_\theta [d(X)]=\int_{R^n}d(x)f(x|\theta)dx$$ Then

\begin{align*} h'(\theta)=\int_{R^n}d(x)\frac{\partial f(x|\theta)}{\partial \theta}dx&=\int_{R^n}d(x)\frac{\partial f(x|\theta)}{\partial \theta}\frac{f(x|\theta)}{f(x|\theta)}dx\\&=\int_{R^n}d(x)\frac{\partial \log(f(x|\theta))}{\partial \theta}f(x|\theta)dx \tag{1} \end{align*}

i.e. $(1)$ is

$$h'(\theta)=E_\theta\left [d(x)\frac{\partial \log(f(x|\theta))}{\partial \theta}\right] \tag{14.14}$$

Therefore, if you want to calculate $h'(\theta)$ just calculate directly by $(1)$ but you need to know the distribution of $X$

Ok, since you update the question, and $X$ has a Bernoulli distrbituion and your $d(X)$ is the $\bar{X}$

Then why not directly calculte the $h'(\theta)$ by the definiton of $(14.12)$



| cite | improve this answer | |
  • $\begingroup$ Sorry it is coming $\frac{1}{n}$ $\endgroup$ – str Nov 16 '17 at 11:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.