0
$\begingroup$

Multivariate normal distribution [edit] The FIM for a $N$-variate multivariate normal distribution, $X \sim N(\mu(\theta), \Sigma(\theta))$ has a special form. Let the $K$-dimensional vector of parameters be $\theta=\left[\begin{array}{lll}\theta_{1} & \ldots & \theta_{K}\end{array}\right]^{\top}$ and the vector of random normal variables be $X=\left[\begin{array}{lll}X_{1} & \ldots & X_{N}\end{array}\right]^{\top} .$ Assume that the mean values of these random variables are $\mu(\theta)=\left[\mu_{1}(\theta) \quad \ldots \quad \mu_{N}(\theta)\right]^{\top}$, and let $\Sigma(\theta)$ be the covariance matrix. Then, for $1 \leq m, n \leq K$, the $(m, n)$ entry of the FIM is: $^{[16]}$ $$ \mathcal{I}_{m, n}=\frac{\partial \mu^{\top}}{\partial \theta_{m}} \Sigma^{-1} \frac{\partial \mu}{\partial \theta_{n}}+\frac{1}{2} \operatorname{tr}\left(\Sigma^{-1} \frac{\partial \Sigma}{\partial \theta_{m}} \Sigma^{-1} \frac{\partial \Sigma}{\partial \theta_{n}}\right) $$ where $(\cdot)^{\top}$ denotes the transpose of a vector, $\operatorname{tr}(\cdot)$ denotes the trace of a square matrix, and: $$ \begin{aligned} \frac{\partial \mu}{\partial \theta_{m}} &=\left[\begin{array}{lll} \frac{\partial \mu_{1}}{\partial \theta_{m}} & \frac{\partial \mu_{2}}{\partial \theta_{m}} & \cdots & \frac{\partial \mu_{N}}{\partial \theta_{m}} \end{array}\right]^{\top} \\ \frac{\partial \Sigma}{\partial \theta_{m}} &=\left[\begin{array}{cccc} \frac{\partial \Sigma_{1,1}}{\partial \theta_{m}} & \frac{\partial \Sigma_{1,2}}{\partial \theta_{m}} & \cdots & \frac{\partial \Sigma_{1, N}}{\partial \theta_{m}} \\ \frac{\partial \Sigma_{2,1}}{\partial \theta_{m}} & \frac{\partial \Sigma_{2,2}}{\partial \theta_{m}} & \cdots & \frac{\partial \Sigma_{2, N}}{\partial \theta_{m}} \\ \vdots & \vdots & \ddots & \vdots \\ \frac{\partial \Sigma_{N, 1}}{\partial \theta_{m}} & \frac{\partial \Sigma_{N, 2}}{\partial \theta_{m}} & \cdots & \frac{\partial \Sigma_{N, N}}{\partial \theta_{m}} \end{array}\right] \end{aligned} $$

Most importantly: What is the variable "m" in the definition of the multivariate normal Fisher Information??

This Wikipedia Definition does not make sense as the fisher information is a metric tensor induced by the hypothesis space and therefore is guaranteed to be symmetric. Writting each entry of the fisher information as $\mathcal{I}_{m,n}$ where $1\leq m, n\leq K$ suggests the Fisher Information Matrix is not symmetric. We do not even have an upper bound on "m"!

Also Is K, the # parameters equal to the N, the number of different mean variables?

What's supposed to be the dimensions of the Fisher info for a multivariate normal?

Source: https://en.wikipedia.org/wiki/Fisher_information

$\endgroup$
2
  • 1
    $\begingroup$ I suppose that first part is quoted from Wikipedia, so better to quote it. Also, put the question in the question body. $\endgroup$
    – Firebug
    Sep 29, 2021 at 12:17
  • $\begingroup$ Is "m" supposed to be the number of mean variables? I think you should have one mean variable per every variance variable . $\endgroup$
    – user318514
    Sep 29, 2021 at 12:25

1 Answer 1

0
$\begingroup$

m is an index into the parameters.

Yes you do have an upper bound on m. The notation you show means that both m and n are bounded by K.

K is the number of parameters, which has nothing to do with the number of mean variables.

The dimension is given in the notation: it's a square matrix.

$\endgroup$
10
  • $\begingroup$ The unknown parameters in a multivariate standard normal are the means and the squared standard deviations. Is it true that K=# unknown means/2=#unknown standard deviations squared/2? If K=#unknown means/2, then K is related with # mean variables. I cannot imagine a case where you know some of the means, and none of the standard deviations squared, and therefore K!= unknown-means/2. $\endgroup$
    – user318514
    Sep 29, 2021 at 13:32
  • $\begingroup$ @Germania No, K is the number of parameters in the model. $\endgroup$
    – Neil G
    Sep 29, 2021 at 13:39
  • $\begingroup$ A parameter is simply what you do not know and have to estimate. By definition of parameter, a mean could be unknown and is estimated, and so is a squared standard deviation. The Multivariate NormalModel has mus and sigmas unknown. $\endgroup$
    – user318514
    Sep 29, 2021 at 13:39
  • $\begingroup$ @Germania You have it right there in your question. K is the length of $\theta$. $\endgroup$
    – Neil G
    Sep 29, 2021 at 13:40
  • $\begingroup$ I don't understand why the means are a function of $\theta$. The means themselves are examples of $\theta$? $\endgroup$
    – user318514
    Sep 29, 2021 at 13:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.