# Statistical distance between two matrices

• The statistical distance between two probability distributions can be measured with $$f$$-divergences such as the KL-divergence.
• The statistical distance between two clusters can be measured with distance metrics.

How can the statistical distance between two matrices be measured? matrix $$A\in \mathbb{R}^{n\times p}$$ and matrix $$B\in \mathbb{R}^{n\times p}$$ have the same dimensions, but more interest in square and symmetric matrices, $$n=p$$.

A previous suggestion was the norm of the matrix-difference, but without much reasoning or proof of usage. The term 'distance matrix' was found, but not sure if that applies to this problem.

• Is it inadequate to flatten each matrix to a long vector and the do KL divergence?
– Dave
Sep 14, 2020 at 3:40
• might not be good for matrices that have a characteristic diagonal Sep 14, 2020 at 3:40
• what is "statistical" about the matrices you are measuring distance between? Sep 14, 2020 at 4:33
• Are you thinking about these matrices as functions from $\mathbb{R}^n$ to $\mathbb{R}^p$ or as collections of $np$ numbers or as $n$ realisations of a $p$-vector or as covariance matrices or something else? It's going to make a difference as to what sort of distances you care about and what the randomness looks like Sep 14, 2020 at 6:00
• A metric (the formal concept of "distance") is equivalent to the norm of the difference if that metric is absolutely homogeneous and translation invariant. To add to @ThomasLumley's comment, there are infinitely many metrics you can put on the space $\mathbb{R}^{n \times p}$, so which one you should choose depends on what you are doing. But if your application justifies using an absolutely homogeneous and translation-invariant metric, then it will be (provably) the norm of a matrix difference, see en.wikipedia.org/wiki/… Sep 14, 2020 at 9:41

The natural matrix analogue for histograms or probabilty distributions are density matrices, i.e. symmetric, positive semi-definite matrices whose trace sums to $$1$$. They predominently occur in information theory and quantum mechanics. For these matrices plenty of the classical concepts can be defined in a similar fashion. The entropy of such a matrix, say $$A$$, goes under 'von Neumann entropy' and reads $$\begin{equation} S(A)=-\operatorname{tr}(A\log A). \end{equation}$$ The quantum relative entropy between two density matrices $$A$$ and $$B$$ is given by $$\begin{equation} S(A|B)=-\operatorname{tr}(A\log B)-S(A)=\operatorname{tr}(A(\log A -\log B)). \end{equation}$$ There do exist also Wasserstein distance concepts for such density matrices.