0
$\begingroup$

Multivariate normal distribution [edit] The FIM for a $N$-variate multivariate normal distribution, $X \sim N(\mu(\theta), \Sigma(\theta))$ has a special form. Let the $K$-dimensional vector of parameters be $\theta=\left[\begin{array}{lll}\theta_{1} & \ldots & \theta_{K}\end{array}\right]^{\top}$ and the vector of random normal variables be $X=\left[\begin{array}{lll}X_{1} & \ldots & X_{N}\end{array}\right]^{\top} .$ Assume that the mean values of these random variables are $\mu(\theta)=\left[\mu_{1}(\theta) \quad \ldots \quad \mu_{N}(\theta)\right]^{\top}$, and let $\Sigma(\theta)$ be the covariance matrix. Then, for $1 \leq m, n \leq K$, the $(m, n)$ entry of the FIM is: $^{[16]}$ $$ \mathcal{I}_{m, n}=\frac{\partial \mu^{\top}}{\partial \theta_{m}} \Sigma^{-1} \frac{\partial \mu}{\partial \theta_{n}}+\frac{1}{2} \operatorname{tr}\left(\Sigma^{-1} \frac{\partial \Sigma}{\partial \theta_{m}} \Sigma^{-1} \frac{\partial \Sigma}{\partial \theta_{n}}\right) $$ where $(\cdot)^{\top}$ denotes the transpose of a vector, $\operatorname{tr}(\cdot)$ denotes the trace of a square matrix, and: $$ \begin{aligned} \frac{\partial \mu}{\partial \theta_{m}} &=\left[\begin{array}{lll} \frac{\partial \mu_{1}}{\partial \theta_{m}} & \frac{\partial \mu_{2}}{\partial \theta_{m}} & \cdots & \frac{\partial \mu_{N}}{\partial \theta_{m}} \end{array}\right]^{\top} \\ \frac{\partial \Sigma}{\partial \theta_{m}} &=\left[\begin{array}{cccc} \frac{\partial \Sigma_{1,1}}{\partial \theta_{m}} & \frac{\partial \Sigma_{1,2}}{\partial \theta_{m}} & \cdots & \frac{\partial \Sigma_{1, N}}{\partial \theta_{m}} \\ \frac{\partial \Sigma_{2,1}}{\partial \theta_{m}} & \frac{\partial \Sigma_{2,2}}{\partial \theta_{m}} & \cdots & \frac{\partial \Sigma_{2, N}}{\partial \theta_{m}} \\ \vdots & \vdots & \ddots & \vdots \\ \frac{\partial \Sigma_{N, 1}}{\partial \theta_{m}} & \frac{\partial \Sigma_{N, 2}}{\partial \theta_{m}} & \cdots & \frac{\partial \Sigma_{N, N}}{\partial \theta_{m}} \end{array}\right] \end{aligned} $$

Most importantly: What is the variable "m" in the definition of the multivariate normal Fisher Information??

This Wikipedia Definition does not make sense as the fisher information is a metric tensor induced by the hypothesis space and therefore is guaranteed to be symmetric. Writting each entry of the fisher information as $\mathcal{I}_{m,n}$ where $1\leq m, n\leq K$ suggests the Fisher Information Matrix is not symmetric. We do not even have an upper bound on "m"!

Also Is K, the # parameters equal to the N, the number of different mean variables?

What's supposed to be the dimensions of the Fisher info for a multivariate normal?

Source: https://en.wikipedia.org/wiki/Fisher_information

$\endgroup$
2
  • 1
    $\begingroup$ I suppose that first part is quoted from Wikipedia, so better to quote it. Also, put the question in the question body. $\endgroup$
    – Firebug
    Sep 29 '21 at 12:17
  • $\begingroup$ Is "m" supposed to be the number of mean variables? I think you should have one mean variable per every variance variable . $\endgroup$
    – Germania
    Sep 29 '21 at 12:25
0
$\begingroup$

m is an index into the parameters.

Yes you do have an upper bound on m. The notation you show means that both m and n are bounded by K.

K is the number of parameters, which has nothing to do with the number of mean variables.

The dimension is given in the notation: it's a square matrix.

$\endgroup$
10
  • $\begingroup$ The unknown parameters in a multivariate standard normal are the means and the squared standard deviations. Is it true that K=# unknown means/2=#unknown standard deviations squared/2? If K=#unknown means/2, then K is related with # mean variables. I cannot imagine a case where you know some of the means, and none of the standard deviations squared, and therefore K!= unknown-means/2. $\endgroup$
    – Germania
    Sep 29 '21 at 13:32
  • $\begingroup$ @Germania No, K is the number of parameters in the model. $\endgroup$
    – Neil G
    Sep 29 '21 at 13:39
  • $\begingroup$ A parameter is simply what you do not know and have to estimate. By definition of parameter, a mean could be unknown and is estimated, and so is a squared standard deviation. The Multivariate NormalModel has mus and sigmas unknown. $\endgroup$
    – Germania
    Sep 29 '21 at 13:39
  • $\begingroup$ @Germania You have it right there in your question. K is the length of $\theta$. $\endgroup$
    – Neil G
    Sep 29 '21 at 13:40
  • $\begingroup$ I don't understand why the means are a function of $\theta$. The means themselves are examples of $\theta$? $\endgroup$
    – Germania
    Sep 29 '21 at 13:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.