2
$\begingroup$

With $\textbf{F}(\theta)$ denoting the Fisher matrix, $\textbf{V}(\theta)$ the variance matrix, $\textbf{C}(\theta)$ the covariance matrix, and parameter vector $\theta = [\theta_1, \theta_2, \dots, \theta_N]$. It is well known that the covariance matrix

$$\textbf{C}(\theta) \ge \textbf{F}^{-1}(\theta) ~~~~~~~(\star)$$

Also for a single parameter $\theta_i$ we know

$$\textbf{V}(\theta_i) \ge \frac{1}{\textbf{F}_{ii}} ~~~~~~~(\star \star)$$

But from $(\star)$ $$\textbf{C}(\theta_i) \ge (\textbf{F}^{-1})_{ii} ~~~~~~~(\star \star \star)$$

Given that from linear algebra $$(\textbf{F}^{-1})_{ii} \ge \frac{1}{\textbf{F}_{ii}},$$

how can we reconcile $(\star \star)$ and $(\star \star \star)$?

$\endgroup$

1 Answer 1

1
$\begingroup$

Foreword: I take it for granted that the preliminary conditions for the Cramér–Rao bound are satisfied. You always should validate it prior to usage.

The CRB for the general univariate case is $$Var\left(\hat{\theta}\right)\geq \frac{\left(1+\frac{\partial}{\partial\theta}b\left(\hat{\theta}\right)\right)^2}{I(\theta)},$$

with $b\left(\hat{\theta}\right)$ being the bias function. If $\hat{\theta}$ is unbiased then $b\left(\hat{\theta}\right)=0$ and the numerator is 1.

Let $\psi(\theta)$ be an unbiased estimator of the vector $\theta$, the CRB of the multivariate unbiased case is

$$Cov\left(\psi(\theta)\right)\geq I(\theta)^{-1},$$

and for the estimator of $\theta_i$ you get

$$Var\left(\psi(\theta)_i\right)\geq \left( I(\theta)^{-1} \right)_{ii}.$$

The loose lower bound for reciprocal of diagonal is $\left( I(\theta)^{-1} \right)_{ii} \geq \left( I(\theta)_{ii}\right)^{-1} $ so overall you get

$$Var\left(\psi(\theta)_i\right)\geq \left( I(\theta)^{-1} \right)_{ii}\geq \left( I(\theta)_{ii}\right)^{-1}.$$

The univariate case has the slightly different same bound because you have no problems of taking the reciprocal of a scalar. Overall, regarding the title of your questions - both bounds are correct, one of them is tighter.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.