0
$\begingroup$

I have been researching about the Cramer-Rao bound and I have found two inequalities:

$$\text{Var}\left(\hat{\theta}\right)\geq\frac{1}{\text{E}\left[\left[\frac{\partial}{\partial\theta}\ln f(X;\theta)\right]^2\right]}$$

$$\text{Var}\left(\hat{\theta}\right)\geq\frac{1}{n\text{E}\left[\left[\frac{\partial}{\partial\theta}\ln f(X;\theta)\right]^2\right]}$$

It is easy to see that the second is more general than the first since:

$$\frac{1}{\text{E}\left[\left[\frac{\partial}{\partial\theta}\ln f(X;\theta)\right]^2\right]}\geq\frac{1}{n\text{E}\left[\left[\frac{\partial}{\partial\theta}\ln f(X;\theta)\right]^2\right]}$$

I would like to know why there are two bounds. In other words I want to know when I can't use the first one and have to use the second bound.

$\endgroup$
1
  • $\begingroup$ Well, first notice that when $n$ = 1, the two expressions for the CR bound are equivalent. Also, the expressions are written using log probabilities base $e$ (via the pdf) instead of the natural log likelihood. The natural log of the product becomes the sum of the individual natural logs for the variables/parameters making up the pdf. So, in the end, the answer to your question depends on whether you are working with a probability or a likelihood. $\endgroup$ Commented Mar 14, 2021 at 5:40

1 Answer 1

0
$\begingroup$

They are really equivalent. The leftmost expression assumes a single realization $x$ distributed according to $X$. The rightmost expression is appropriate to "convert" from the probability scale to the likelihood scale.

That is,

$L(\theta | X) = L(\theta | x_1,...x_n) = \prod_{i=1}^n{f(x_i | \theta)}$.

Taking natural logs (ln) on both sides of the above expression gives

$l(\theta | X)$ = ln$\left(\prod_{i=1}^n{f(x_i | \theta)}\right)$ = $\sum_{i=1}^n{lnf(x_i | \theta)}$

Note: As an aside, for the Cramer Rao lower bound, we typically like to work with the quantity

$-\mathbb{E}\left[\frac{\partial^2}{\partial \theta^2}lnf(x_i | \theta)\right]$

instead of

$\mathbb{E}\left[(\frac{\partial}{\partial \theta}lnf(x_i | \theta))^2\right]$

$\endgroup$
1
  • $\begingroup$ Thanks a lot. I finally understand my mistake. $\endgroup$
    – KKLK
    Commented Mar 15, 2021 at 3:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.