# Is there a multi-Gaussian version of the Mahalanobis distance ?

Let's say we are in an $N$-dimensional space and that we have a large set of data. The distribution of this $N$-dimensional point cloud can be modeled by a multivariate Gaussian mixture model (estimated using the EM algorithm).

Now, given two data points $x$ and $y$ from my dataset, which distance can-I use to evaluate the closeness of these two points given the global distribution ?

In the case of a single Gaussian, I think that the Mahalanobis distance would be a natural choice, but when the global distribution is known to be non-Gaussian and a GMM is used to model the data, we end-up with $K$ different covariance matrices (one for each Gaussian). Should I compute $K$ Mahlanobis distances (one for each covariance matrix) then do a weighted sum using the posterior probabilities over the Gaussians ? Something like this maybe :

$$dist(x,y)=\sum_{i=1}^K \frac{\gamma_i(x)+\gamma_i(y)}{2} \sqrt{(x-y)^T\Sigma_i^{-1}(x-y)}$$

where $\Sigma_i$ is the covariance matrix of the $i$-th Gaussian and $\gamma_i(x)$ is the posterior probability of the vector $x$ with respect to the Gaussian $i$.

Any suggestions ? Thanks !

• What do you want to use your "distance" for, and at what phase of the modeling, "solution", or model results analysis process do you wish to use it? There are a lot of papers floating around addressing use of Mahalanobis Distance within multiple cluster situations, such as mixture models. A little Googling may serve you well. Commented May 10, 2016 at 17:14
• This might be a good question. If you explain why you need it. Why, for example, just a Euclidean distance won't suit... You see, Mahalanobis distance is Euclidean distance corrected for the "curvature" (so to speak) of the space induced by the correlatedness of the dimensions. With more than one gaussians, more than one "curved" alternative spaces exist. In this context, what is your purpose then? Commented May 10, 2016 at 17:19
• Thanks for your responses. Actually, I want to use topological data analysis (TDA) to analyse a large amount of data and such techniques rely on "lenses" which are real valued functions used to described data. Based on this measure, a simplicial complex is built to describe the topological stricture of data. After a little research, I found what I was looking for, a paper called "Deriving cluster analytic distance functions from gaussian mixture models" which proposes Mahalanobis-like distance for GMMs. Commented May 11, 2016 at 8:52

## 2 Answers

After a little research, I found what I was looking for, a paper called "Deriving cluster analytic distance functions from gaussian mixture models" which proposes an extension of the Mahalanobis distance in the context of multi-modal data (a GMM representation) using Fisher Kernel method and other techniques.

I know this is several years ago, but I wanted to point out that it is possible (given some data) to estimate the Kullback-leibler divergence between two GMMs. Depending on your goal, this may prove a very elegant approach.