# Statistical distance between two matrices

• The statistical distance between two probability distributions can be measured with $$f$$-divergences such as the KL-divergence.
• The statistical distance between two clusters can be measured with distance metrics.

How can the statistical distance between two matrices be measured? matrix $$A\in \mathbb{R}^{n\times p}$$ and matrix $$B\in \mathbb{R}^{n\times p}$$ have the same dimensions, but more interest in square and symmetric matrices, $$n=p$$.

A previous suggestion was the norm of the matrix-difference, but without much reasoning or proof of usage. The term 'distance matrix' was found, but not sure if that applies to this problem.

• Is it inadequate to flatten each matrix to a long vector and the do KL divergence? – Dave Sep 14 '20 at 3:40
• might not be good for matrices that have a characteristic diagonal – develarist Sep 14 '20 at 3:40
• what is "statistical" about the matrices you are measuring distance between? – shimao Sep 14 '20 at 4:33
• Are you thinking about these matrices as functions from $\mathbb{R}^n$ to $\mathbb{R}^p$ or as collections of $np$ numbers or as $n$ realisations of a $p$-vector or as covariance matrices or something else? It's going to make a difference as to what sort of distances you care about and what the randomness looks like – Thomas Lumley Sep 14 '20 at 6:00
• A metric (the formal concept of "distance") is equivalent to the norm of the difference if that metric is absolutely homogeneous and translation invariant. To add to @ThomasLumley's comment, there are infinitely many metrics you can put on the space $\mathbb{R}^{n \times p}$, so which one you should choose depends on what you are doing. But if your application justifies using an absolutely homogeneous and translation-invariant metric, then it will be (provably) the norm of a matrix difference, see en.wikipedia.org/wiki/… – Eric Perkerson Sep 14 '20 at 9:41

## 1 Answer

The natural matrix analogue for histograms or probabilty distributions are density matrices, i.e. symmetric, positive semi-definite matrices whose trace sums to $$1$$. They predominently occur in information theory and quantum mechanics. For these matrices plenty of the classical concepts can be defined in a similar fashion. The entropy of such a matrix, say $$A$$, goes under 'von Neumann entropy' and reads $$\begin{equation} S(A)=-\operatorname{tr}(A\log A). \end{equation}$$ The quantum relative entropy between two density matrices $$A$$ and $$B$$ is given by $$\begin{equation} S(A|B)=-\operatorname{tr}(A\log B)-S(A)=\operatorname{tr}(A(\log A -\log B)). \end{equation}$$ There do exist also Wasserstein distance concepts for such density matrices.

• Could you include wasserstein density matrices in your answer – develarist Nov 5 '20 at 4:09
• You can find tons of papers on quantum Wasserstein distance, i.e. hal.archives-ouvertes.fr/hal-02214344v3/document. – Tobsn Nov 5 '20 at 7:31
• your answer points out a possible substitute for probability vectors and histograms, the density matrix, but the density matrix only contain pure and mixed states, which don't have anything to do with the question. The question in fact isn't about probability distributions, it's purely about matrices containing elements, so I don't see how probabilities suddenly became the topic. The only mention of "distance" is finally brought up in regard to the Wasserstein distance between these density matrices you've proposed, but nothing else is given besides a link for me to hunt the answer myself. – develarist Nov 5 '20 at 7:53
• No, you were asking for 'statistical distances'. And such a priori are only defined for probability-alike objects like histograms or probability densities. For matrices in general, it makes no sense to ask for statistical distances. Yet there are certain classes of matrices for which one can define analoges. I pointed out one of them. – Tobsn Nov 5 '20 at 8:51
• don't worry, I was asking for statistical distances between matrices specifically – develarist Nov 5 '20 at 8:56