1
$\begingroup$
  • The statistical distance between two probability distributions can be measured with $f$-divergences such as the KL-divergence.
  • The statistical distance between two clusters can be measured with distance metrics.

How can the statistical distance between two matrices be measured? matrix $A\in \mathbb{R}^{n\times p}$ and matrix $B\in \mathbb{R}^{n\times p}$ have the same dimensions, but more interest in square and symmetric matrices, $n=p$.

A previous suggestion was the norm of the matrix-difference, but without much reasoning or proof of usage. The term 'distance matrix' was found, but not sure if that applies to this problem.

Improve this question
$\endgroup$
6
  • $\begingroup$ Is it inadequate to flatten each matrix to a long vector and the do KL divergence? $\endgroup$
    – Dave
    Sep 14, 2020 at 3:40
  • $\begingroup$ might not be good for matrices that have a characteristic diagonal $\endgroup$
    – develarist
    Sep 14, 2020 at 3:40
  • 1
    $\begingroup$ what is "statistical" about the matrices you are measuring distance between? $\endgroup$
    – shimao
    Sep 14, 2020 at 4:33
  • 2
    $\begingroup$ Are you thinking about these matrices as functions from $\mathbb{R}^n$ to $\mathbb{R}^p$ or as collections of $np$ numbers or as $n$ realisations of a $p$-vector or as covariance matrices or something else? It's going to make a difference as to what sort of distances you care about and what the randomness looks like $\endgroup$
    – Thomas Lumley
    Sep 14, 2020 at 6:00
  • 1
    $\begingroup$ A metric (the formal concept of "distance") is equivalent to the norm of the difference if that metric is absolutely homogeneous and translation invariant. To add to @ThomasLumley's comment, there are infinitely many metrics you can put on the space $\mathbb{R}^{n \times p}$, so which one you should choose depends on what you are doing. But if your application justifies using an absolutely homogeneous and translation-invariant metric, then it will be (provably) the norm of a matrix difference, see en.wikipedia.org/wiki/… $\endgroup$
    – Eric Perkerson
    Sep 14, 2020 at 9:41

1 Answer 1

Reset to default
1
$\begingroup$

The natural matrix analogue for histograms or probabilty distributions are density matrices, i.e. symmetric, positive semi-definite matrices whose trace sums to $1$. They predominently occur in information theory and quantum mechanics. For these matrices plenty of the classical concepts can be defined in a similar fashion. The entropy of such a matrix, say $A$, goes under 'von Neumann entropy' and reads \begin{equation} S(A)=-\operatorname{tr}(A\log A). \end{equation} The quantum relative entropy between two density matrices $A$ and $B$ is given by \begin{equation} S(A|B)=-\operatorname{tr}(A\log B)-S(A)=\operatorname{tr}(A(\log A -\log B)). \end{equation} There do exist also Wasserstein distance concepts for such density matrices.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Could you include wasserstein density matrices in your answer $\endgroup$
    – develarist
    Nov 5, 2020 at 4:09
  • $\begingroup$ You can find tons of papers on quantum Wasserstein distance, i.e. hal.archives-ouvertes.fr/hal-02214344v3/document. $\endgroup$
    – Tobsn
    Nov 5, 2020 at 7:31
  • $\begingroup$ your answer points out a possible substitute for probability vectors and histograms, the density matrix, but the density matrix only contain pure and mixed states, which don't have anything to do with the question. The question in fact isn't about probability distributions, it's purely about matrices containing elements, so I don't see how probabilities suddenly became the topic. The only mention of "distance" is finally brought up in regard to the Wasserstein distance between these density matrices you've proposed, but nothing else is given besides a link for me to hunt the answer myself. $\endgroup$
    – develarist
    Nov 5, 2020 at 7:53
  • $\begingroup$ No, you were asking for 'statistical distances'. And such a priori are only defined for probability-alike objects like histograms or probability densities. For matrices in general, it makes no sense to ask for statistical distances. Yet there are certain classes of matrices for which one can define analoges. I pointed out one of them. $\endgroup$
    – Tobsn
    Nov 5, 2020 at 8:51
  • $\begingroup$ don't worry, I was asking for statistical distances between matrices specifically $\endgroup$
    – develarist
    Nov 5, 2020 at 8:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.