# What is the definition and upper bound on the variable "m" in the definition of the multivariate normal Fisher Information?

Multivariate normal distribution  The FIM for a $$N$$-variate multivariate normal distribution, $$X \sim N(\mu(\theta), \Sigma(\theta))$$ has a special form. Let the $$K$$-dimensional vector of parameters be $$\theta=\left[\begin{array}{lll}\theta_{1} & \ldots & \theta_{K}\end{array}\right]^{\top}$$ and the vector of random normal variables be $$X=\left[\begin{array}{lll}X_{1} & \ldots & X_{N}\end{array}\right]^{\top} .$$ Assume that the mean values of these random variables are $$\mu(\theta)=\left[\mu_{1}(\theta) \quad \ldots \quad \mu_{N}(\theta)\right]^{\top}$$, and let $$\Sigma(\theta)$$ be the covariance matrix. Then, for $$1 \leq m, n \leq K$$, the $$(m, n)$$ entry of the FIM is: $$^{[16]}$$ $$\mathcal{I}_{m, n}=\frac{\partial \mu^{\top}}{\partial \theta_{m}} \Sigma^{-1} \frac{\partial \mu}{\partial \theta_{n}}+\frac{1}{2} \operatorname{tr}\left(\Sigma^{-1} \frac{\partial \Sigma}{\partial \theta_{m}} \Sigma^{-1} \frac{\partial \Sigma}{\partial \theta_{n}}\right)$$ where $$(\cdot)^{\top}$$ denotes the transpose of a vector, $$\operatorname{tr}(\cdot)$$ denotes the trace of a square matrix, and: \begin{aligned} \frac{\partial \mu}{\partial \theta_{m}} &=\left[\begin{array}{lll} \frac{\partial \mu_{1}}{\partial \theta_{m}} & \frac{\partial \mu_{2}}{\partial \theta_{m}} & \cdots & \frac{\partial \mu_{N}}{\partial \theta_{m}} \end{array}\right]^{\top} \\ \frac{\partial \Sigma}{\partial \theta_{m}} &=\left[\begin{array}{cccc} \frac{\partial \Sigma_{1,1}}{\partial \theta_{m}} & \frac{\partial \Sigma_{1,2}}{\partial \theta_{m}} & \cdots & \frac{\partial \Sigma_{1, N}}{\partial \theta_{m}} \\ \frac{\partial \Sigma_{2,1}}{\partial \theta_{m}} & \frac{\partial \Sigma_{2,2}}{\partial \theta_{m}} & \cdots & \frac{\partial \Sigma_{2, N}}{\partial \theta_{m}} \\ \vdots & \vdots & \ddots & \vdots \\ \frac{\partial \Sigma_{N, 1}}{\partial \theta_{m}} & \frac{\partial \Sigma_{N, 2}}{\partial \theta_{m}} & \cdots & \frac{\partial \Sigma_{N, N}}{\partial \theta_{m}} \end{array}\right] \end{aligned}

Most importantly: What is the variable "m" in the definition of the multivariate normal Fisher Information??

This Wikipedia Definition does not make sense as the fisher information is a metric tensor induced by the hypothesis space and therefore is guaranteed to be symmetric. Writting each entry of the fisher information as $$\mathcal{I}_{m,n}$$ where $$1\leq m, n\leq K$$ suggests the Fisher Information Matrix is not symmetric. We do not even have an upper bound on "m"!

Also Is K, the # parameters equal to the N, the number of different mean variables?

What's supposed to be the dimensions of the Fisher info for a multivariate normal?

• I suppose that first part is quoted from Wikipedia, so better to quote it. Also, put the question in the question body. Sep 29 '21 at 12:17
• Is "m" supposed to be the number of mean variables? I think you should have one mean variable per every variance variable . Sep 29 '21 at 12:25

m is an index into the parameters.

Yes you do have an upper bound on m. The notation you show means that both m and n are bounded by K.

K is the number of parameters, which has nothing to do with the number of mean variables.

The dimension is given in the notation: it's a square matrix.

• The unknown parameters in a multivariate standard normal are the means and the squared standard deviations. Is it true that K=# unknown means/2=#unknown standard deviations squared/2? If K=#unknown means/2, then K is related with # mean variables. I cannot imagine a case where you know some of the means, and none of the standard deviations squared, and therefore K!= unknown-means/2. Sep 29 '21 at 13:32
• @Germania No, K is the number of parameters in the model. Sep 29 '21 at 13:39
• A parameter is simply what you do not know and have to estimate. By definition of parameter, a mean could be unknown and is estimated, and so is a squared standard deviation. The Multivariate NormalModel has mus and sigmas unknown. Sep 29 '21 at 13:39
• @Germania You have it right there in your question. K is the length of $\theta$. Sep 29 '21 at 13:40
• I don't understand why the means are a function of $\theta$. The means themselves are examples of $\theta$? Sep 29 '21 at 13:41