0
$\begingroup$

I have been researching about the Cramer-Rao bound and I have found two inequalities:

$$\text{Var}\left(\hat{\theta}\right)\geq\frac{1}{\text{E}\left[\left[\frac{\partial}{\partial\theta}\ln f(X;\theta)\right]^2\right]}$$

$$\text{Var}\left(\hat{\theta}\right)\geq\frac{1}{n\text{E}\left[\left[\frac{\partial}{\partial\theta}\ln f(X;\theta)\right]^2\right]}$$

It is easy to see that the second is more general than the first since:

$$\frac{1}{\text{E}\left[\left[\frac{\partial}{\partial\theta}\ln f(X;\theta)\right]^2\right]}\geq\frac{1}{n\text{E}\left[\left[\frac{\partial}{\partial\theta}\ln f(X;\theta)\right]^2\right]}$$

I would like to know why there are two bounds. In other words I want to know when I can't use the first one and have to use the second bound.

$\endgroup$
1
  • $\begingroup$ Well, first notice that when $n$ = 1, the two expressions for the CR bound are equivalent. Also, the expressions are written using log probabilities base $e$ (via the pdf) instead of the natural log likelihood. The natural log of the product becomes the sum of the individual natural logs for the variables/parameters making up the pdf. So, in the end, the answer to your question depends on whether you are working with a probability or a likelihood. $\endgroup$ Mar 14, 2021 at 5:40

1 Answer 1

0
$\begingroup$

They are really equivalent. The leftmost expression assumes a single realization $x$ distributed according to $X$. The rightmost expression is appropriate to "convert" from the probability scale to the likelihood scale.

That is,

$L(\theta | X) = L(\theta | x_1,...x_n) = \prod_{i=1}^n{f(x_i | \theta)}$.

Taking natural logs (ln) on both sides of the above expression gives

$l(\theta | X)$ = ln$\left(\prod_{i=1}^n{f(x_i | \theta)}\right)$ = $\sum_{i=1}^n{lnf(x_i | \theta)}$

Note: As an aside, for the Cramer Rao lower bound, we typically like to work with the quantity

$-\mathbb{E}\left[\frac{\partial^2}{\partial \theta^2}lnf(x_i | \theta)\right]$

instead of

$\mathbb{E}\left[(\frac{\partial}{\partial \theta}lnf(x_i | \theta))^2\right]$

$\endgroup$
1
  • $\begingroup$ Thanks a lot. I finally understand my mistake. $\endgroup$
    – KKLK
    Mar 15, 2021 at 3:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.