5
$\begingroup$

Let's say we are in an $N$-dimensional space and that we have a large set of data. The distribution of this $N$-dimensional point cloud can be modeled by a multivariate Gaussian mixture model (estimated using the EM algorithm).

Now, given two data points $x$ and $y$ from my dataset, which distance can-I use to evaluate the closeness of these two points given the global distribution ?

In the case of a single Gaussian, I think that the Mahalanobis distance would be a natural choice, but when the global distribution is known to be non-Gaussian and a GMM is used to model the data, we end-up with $K$ different covariance matrices (one for each Gaussian). Should I compute $K$ Mahlanobis distances (one for each covariance matrix) then do a weighted sum using the posterior probabilities over the Gaussians ? Something like this maybe :

\begin{equation} dist(x,y)=\sum_{i=1}^K \frac{\gamma_i(x)+\gamma_i(y)}{2} \sqrt{(x-y)^T\Sigma_i^{-1}(x-y)} \end{equation}

where $\Sigma_i$ is the covariance matrix of the $i$-th Gaussian and $\gamma_i(x)$ is the posterior probability of the vector $x$ with respect to the Gaussian $i$.

Any suggestions ? Thanks !

$\endgroup$
  • 2
    $\begingroup$ What do you want to use your "distance" for, and at what phase of the modeling, "solution", or model results analysis process do you wish to use it? There are a lot of papers floating around addressing use of Mahalanobis Distance within multiple cluster situations, such as mixture models. A little Googling may serve you well. $\endgroup$ – Mark L. Stone May 10 '16 at 17:14
  • 2
    $\begingroup$ This might be a good question. If you explain why you need it. Why, for example, just a Euclidean distance won't suit... You see, Mahalanobis distance is Euclidean distance corrected for the "curvature" (so to speak) of the space induced by the correlatedness of the dimensions. With more than one gaussians, more than one "curved" alternative spaces exist. In this context, what is your purpose then? $\endgroup$ – ttnphns May 10 '16 at 17:19
  • $\begingroup$ Thanks for your responses. Actually, I want to use topological data analysis (TDA) to analyse a large amount of data and such techniques rely on "lenses" which are real valued functions used to described data. Based on this measure, a simplicial complex is built to describe the topological stricture of data. After a little research, I found what I was looking for, a paper called "Deriving cluster analytic distance functions from gaussian mixture models" which proposes Mahalanobis-like distance for GMMs. $\endgroup$ – vphenix May 11 '16 at 8:52
3
$\begingroup$

After a little research, I found what I was looking for, a paper called "Deriving cluster analytic distance functions from gaussian mixture models" which proposes an extension of the Mahalanobis distance in the context of multi-modal data (a GMM representation) using Fisher Kernel method and other techniques.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.