# What is the difference between a matrix normal distribution and the multivariate gaussian distribution?

$$\newcommand{\vec}{\operatorname{vec}}$$Consider a set of $$N$$ matrices $$X_1, X_2, \ldots, X_N$$. I want to estimate the distribution of these matrices represented by the mean and covariance.

I address this problem by simply vectorizing my matrices $$\vec(X_1), \vec(X_2), \ldots, \vec(X_N)$$, then I compute the mean and covariance as I would for a multivariate Gaussian distribution. In other words, this implies that $$\vec(X) \sim \mathcal{N}(M, \Sigma)$$ where $$\vec(X)$$ is reshaped to get $$X$$ after sampling

However, I came across matrix normal distributions, which defines the distribution based on the row covariance and column covariance. So, a matrix sampled from a matrix normal distribution is given by $$X \sim \mathcal{M}\mathcal{N}(M, U, V)$$.

In the wikipedia article, it says that the relation between the matrix normal distribution and the multivariate gaussian distribution for a random matrix $$X$$ is $$X \sim \mathcal{M}\mathcal{N}(M, U, V)$$ if and only if $$X \sim \mathcal{N}(V \otimes U)$$ where $$\otimes$$ is the operator for the kronecker product.

If I generate a set of random matrices, I can estimate the parameters of the distribution in multiple ways. I can vectorize my matrices as treat as a multivariate Gaussian to get the covariance. Or, I can compute the row covariance and column covariance, then I can compute the covariance from the kronecker product.

However, I am not getting the same values for the covariance from the vectorized approach verses the kronecker product approach. Should I get the same result for each? If not, why are they different? I don't understand what these distributions represent in practice, so I not sure which should be used in which scenario.

• How are you estimating $U$ and $V$ from the samples? Are you using iterative maximum likelihood (as presented here en.wikipedia.org/wiki/…)? Are you estimating $\Sigma$ via vanilla maximum likelihood? Commented May 31 at 23:36
• @JohandeAguas I estimate $U$ and $V$ using the iterative approach in the link you provided. If vectoring and representing as a multivariate gaussian, I use vanilla mle. Commented Jun 5 at 20:17
• Check out: Dawid, A. Philip. "Some matrix-variate distribution theory: notational considerations and a Bayesian application." Biometrika 68.1 (1981): 265-274. Commented Jun 20 at 16:09

Example for diagonal matrices $$U = \begin{bmatrix} u_1 & 0 & 0 \\0& u_2 & 0 \\ 0&0&u_3 \end{bmatrix}$$ and $$V = \begin{bmatrix} v_1 & 0 & 0 \\0& v_2 & 0 \\ 0&0&v_3 \end{bmatrix}$$
lead to $$\Sigma = \begin{bmatrix} u_1v_1 & 0 &0&0 &0&0&0&0&0\\ 0&u_1v_2 & 0 &0&0 &0&0&0&0\\ 0&0&u_1v_3 & 0 &0 &0&0&0&0\\ 0&0&0&u_2v_1 & 0 &0&0&0&0\\ 0&0&0&0&u_2v_2 & 0 &0&0 &0\\ 0&0&0&0&0&u_2v_3 & 0 &0 &0\\ 0&0&0&0&0&0&u_3v_1 & 0 &0\\ 0&0&0&0&0&0&0&u_3v_2 & 0 \\ 0&0&0&0&0&0&0&0&u_3v_3 \\ \end{bmatrix}$$
the $$2n$$ values in $$U$$ and $$V$$ will lead to $$n^2$$ values in $$\Sigma$$.
So if you estimate those values in $$\Sigma$$ directly instead of $$U \otimes V$$, the you will have a larger degree of possible values and the estimates will be different.
The same is true for non diagonal matrices. The matrix normal distribution restricts the parameterization of the $$n^4$$ values in $$\Sigma$$ to the $$2n^2$$ values in $$U$$ and $$V$$.
• Thanks for this answer. What are the implications of limiting to $2n^2$ values? I suppose I don't quite understand the practical meaning of the row covariance and column covariance as well as the overall covariance computed from these. For example, this isn't just a compressed version of covariance, but instead, it has a different meaning overall, right? Commented Jun 5 at 20:21