I am taking a course in Statistical Inference. We have to calculate the Cramér-Rao lower bound to determine if different statistics are efficient or not.

I have read that as long as we have an unbiased estimator, the CRLB is just the reciprocal of the Fisher Information. Using this, we were handed a lot of different formulas for the Fisher Information in an exercise session. This was however not covered in too much depth.

What I have so far is the following:

Let $f(\mathbf x | \theta)$ be the joint distribution function for the observed sample, then the Fisher Information is

$$I(\theta) = \text{E} \left[ \left( \frac{d}{d \theta} \log f(\mathbf x | \theta) \right)^2 \right] = - \text{E} \left[ \frac{d^2}{d \theta ^2} \log f(\mathbf x | \theta) \right].$$

If the observations are i.i.d. the Fisher Information can also be written as $$I(\theta) = n \text{E} \left[ \left( \frac{d}{d \theta} \log f(x | \theta) \right)^2 \right] = -n \text{E} \left[ \frac{d^2}{d \theta^2} \log f(x | \theta) \right],$$ where $f(x|\theta)$ is the pdf for each observed variable.

My questions are:

  • Are these formulas correct?
  • Are there any other assumptions besides those mentioned required, and are the mentioned assumptions correct?
  • Are these variants sufficient given that we will only consider unbiased estimators?
  • 1
    $\begingroup$ With some regularity conditions, the definition is the very first of your formulas with $\mathbf x$ usually replaced by the random variable $\mathbf X$. The one with the second derivative does not always hold. For full details, you can see Elements of Large-Sample Theory by Lehmann (page 456 onwards). $\endgroup$ – StubbornAtom Jan 8 at 8:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.