Is there a version of the Mahalanobis distance for matrices?

Question

I'm working on a computer vision problem and I want to use the Mahalanobis distance to cluster image patches (2D matrices having the same dimensions). I haven't been able to find any generalisation up to this point and would prefer not to vectorise my patches and end-up with a huge covariance matrix.

Since the exponent term in the multivariate Gaussian distribution density function is related to the Mahalanobis distance, I looked for a matrix version and I found the Matrix normal distribution:

The probability density function for the random matrix $\mathbf{X}(n\times p)$ that follows the matrix normal distribution $\mathcal{MN}_{n,p}(\mathbf{M}, \mathbf{U}, \mathbf{V})$ has the form:

$p(\mathbf{X}\mid\mathbf{M}, \mathbf{U}, \mathbf{V}) = \frac{\exp\left( -\frac{1}{2} \, \mathrm{tr}\left[ \mathbf{V}^{-1} (\mathbf{X} - \mathbf{M})^{T} \mathbf{U}^{-1} (\mathbf{X} - \mathbf{M}) \right] \right)}{(2\pi)^{np/2} |\mathbf{V}|^{n/2} |\mathbf{U}|^{p/2}} $

where $\mathrm{tr}$ denotes trace and $\mathbf{M}$ is $n\times p$, $\mathbf{U}$ is $n\times n$ and $\mathbf{V}$ is $p\times p$.

Now, my question is : is the trace term a generalisation of the Mahalanobis distance applied on matrices or is there another formulation ?

Answer 1

No. There are metrics that try to build on a similar concept using Wishart distribution. I have seen papers in MRI imaging that use the metrics. See p.16 in this slide deck: https://earth.esa.int/c/document_library/get_file?folderId=409343&name=DLFE-5593.pdf

There's a distance called Riemannian metric for positive definite matrices, that I used in the past to measure the distance of covariance matrices. For instance, look at Eq.13 here: https://hal.archives-ouvertes.fr/hal-00820475/document "Classification of covariance matrices using a Riemannian-based kernel for BCI applications", Alexandre Barachant, Stéphane Bonnet, Marco Congedo, Christian Jutten. I just grabbed the first link in Google, this is not the reference paper on the subject

Answer 2

I'm working on a computer vision problem and I want to use the Mahalanobis distance to cluster image patches (2D matrices having the same dimensions). I haven't been able to find any generalisation up to this point and would prefer not to vectorise my patches and end-up with a huge covariance matrix.

If your "image patches" are what I think they are (i.e. 2D arrays of samples), then mathematically they are vectors, not matrices.

In linear algebra, a vector represents a position or a distance in multidimensional space, such as an array of samples, while a matrix represents a linear map from one vector space to another (or possibly the same one). The fact that vectors are commonly written as one-dimensional arrays, and matrices as two-dimensional arrays, is really more of an arbitrary historical convention.

It's not completely arbitrary, since a vector does of course need to be at least one-dimensional, while a matrix, being essentially a vector of vectors, is naturally represented as an array with twice as many dimensions as a vector. But there are plenty of situations, such as when working with 2D image data, where one does end up with vectors that would be most naturally represented as 2D arrays (implying that matrices mapping such "2D vectors" to other "2D vectors" really should be four-dimensional arrays).

The standard way of handling such situations is to flatten the "2D vectors" into ordinary one-dimensional vectors, and any matrices applied to them into two-dimensional block matrices, and then work with them using ordinary linear algebra. This does tend to yield large and awkward-looking expressions if you try to print them out without first transforming them back into a form better suited for the structure of your data, but that's kind of unavoidable when working with such data.

In any case, the question you should ask yourself is whether your 2D "image patches" somehow represent linear transformations between one-dimensional vectors. If not, then they're not matrices, and any attempt to treat them as matrices will probably end up producing nonsense. Instead, you should treat them as what they are, i.e. presumably as vectors of samples.

And yes, if your patches are large, then covariance matrices (which really are matrices in the linear algebra sense!) between them will probably end up being huge and hard to visualize. Which isn't really that surprising, given that they're fundamentally representing a four-dimensional data set, and humans aren't very good at visualizing 4D data.