As we know, the Mahalanobis distance (MD) is one of the distance metrics for measuring two points in multivariate space. In practice, I can compute Mahalanobis distance between two 1D arrays using Python function like scipy.spatial.distance.mahalanobis. However, is it possible to measure the distance between two high dimensional arrays? For instance, I can have two 64*64*3 arrays, each of which represents an image. Is that possible to compute its Mahalanobis distance?
$\begingroup$
$\endgroup$
1
-
2$\begingroup$ Measures such as mahalanobis don’t work very well for high dimensional data. There are at least two reasons, one is the dimensionally curse where in high dimensions all distances collapse and all point’s become indistinguishable $\endgroup$– AksakalCommented Sep 30, 2021 at 21:22
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
There are two issues: correlation matrix is unknown and in high dimensions Euclidian distances loose discriminating power.
The first issue is simply because estimation of high dimensional correlation matrices is nearly impossible. You'll have to regularize them to a point where it's not clear what's left of them after the procedure. Just imagine how much data you need to estimate 12288x12288/2-12288 correlation values. You need that many and much much more images to estimate it. Even with a few hundred rows we end up in La La Land.
The second issue is that in this many dimensions all images will end up being equally far away from each other. Check this paper out: Aggarwal, Charu & Hinneburg, Alexander & Keim, Daniel. (2002). On the Surprising Behavior of Distance Metric in High-Dimensional Space. Since you're dealing with images, i.e. each pixel is an integer number, maybe using Manhattan distance would work for you
$\begingroup$
$\endgroup$
There are issues with this in high dimensions, but if you’re determined to compute the Mahalanobis distance between images, you can flatten them to $64\times 64\times 3=12288$-vectors and then proceed as usual. Your covariance matrix will be $12288\times 12288$.
Flattening an image is reasonable and, in fact, how you would do many image recognition processing machine learning models, such as logistic regression.
$\begingroup$
$\endgroup$
1
According to Wikipedia, you can do it.
Mahalanobis distance (or "generalized squared interpoint distance" for its squared value) can also be defined as a dissimilarity measure between two random vectors x and y of the same distribution with the covariance matrix S :
-
1$\begingroup$ How do you propose OP estimate the conclave matrix? $\endgroup$– AksakalCommented Sep 30, 2021 at 21:23