16
$\begingroup$

(I posted a similar question on math.se.)

In information geometry, the determinant of the Fisher information matrix is a natural volume form on a statistical manifold, so it has a nice geometrical interpretation. The fact that it appears in the definition of a Jeffreys prior, for example, is linked to its invariance under reparametrizations, which is (imho) a geometrical property.

But what is that determinant in statistics? Does it measure anything meaningful? (For example, I would say that if it is zero, then the parameters are not independent. Does this go any further?)

Also, is there any closed form to compute it, at least in some "easy" cases?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
15
$\begingroup$

In many examples, the inverse of the fisher information matrix is the covariance matrix of the parameter estimates $\hat{\beta}$, exactly or approximately. Often it gives that covariance matrix asymptotically. The determinant of a covariance matrix is often called a generalized variance.

So the determinant of the Fisher information matrix is the inverse of that generalized variance. This can be used in experimental design to find optimal experiments (for parameter estimation). In that context, this is called D-optimality, which has a huge literature. so google for "D-optimal experimental design". In practice, it is often easier to maximize the determinant of the inverse covariance matrix, but that is obviously the same thing as minimizing the determinant of its inverse.

There are also many posts on this site, but few has good answers. Here is one: Experimental (factorial) design not exploiting the variance

Improve this answer
$\endgroup$
3
  • 3
    $\begingroup$ So the followup question is what is the significance of generalized variance. Is it related to linear independence? Also please see this question. $\endgroup$
    – GENIVI-LEARNER
    Commented Mar 31, 2020 at 16:11
  • $\begingroup$ @GENIVI-LEARNER Based on the way it is used in Jeffreys prior $p(\theta)\propto |I(\theta)|^{1/2}=1/\sigma_\theta$ s.t. $\sigma_\theta:=$'generalized standard deviation'. I believe it can be thought of as a "summary statistic" that attempts to describe "the variance" as just one scalar even in a multi-dimensional setting. $\endgroup$
    – profPlum
    Commented Jul 28 at 22:10
  • $\begingroup$ @GENIVI-LEARNER Also it's indirectly related to linear independence. Since it is the volume of the parallel-piped of the basis vectors of the Cov-matrix. The volume is greatest when the Covariance matrix is more orthogonal which implies that the different parameters are varying more independently (in a probabilistic and linear sense). $\endgroup$
    – profPlum
    Commented Jul 29 at 18:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.