Let's say I have a group of clusters. Would you recommend Mahalanobis distance measure for checking if new arrived data belongs to existing clusters or it is an outlier?

Also, would you recommend this distance measure during clustering and in which cases?



This is a multivariate Gaussian:

$$ f(x;\mu,\Sigma) = \frac{1}{\sqrt{(2\pi)^{n}|\Sigma|}}e^{(-\frac{1}{2}(x-\mu)^T\Sigma^{-1}(x-\mu))} $$

Mahalanobis distance is related to the power of the exponential: $$MD = \sqrt{(x-\mu)^T\Sigma^{-1}(x-\mu)}$$

So I would say if your underlying distributions are multivariate gaussians, Mahalanobis distance seems useful. The major problem is estimating the precision matrix $\Sigma^{-1}$ for cases that are high dimensional with few observations.

If you have no choice but to perform automated outlier detection, then there are some nice interpretable qualities about MD. In the univariate case, this normalized distance is equivalent to the number of standard deviations from the mean. If your data is indeed normal, it's common to call the points that are '$n$' standard deviations from the mean as outliers ($ MD > n$). If you choose $n=2$ as the outlier threshold, you would be rejecting points that exceed the 95 percentile of the underlying distribution in the univariate case (approximately).

In the multivariate case, the curse of dimensionality comes into play. If you wanted to keep the 95 percentile rule, you would need to reject data based on the quantile of a $\chi$ distribution, as explained here.

  • $\begingroup$ Thank you very much for your answer, just one more question. Would you use this algorithm in case of time series? As to find which time serie seems as an outlier? $\endgroup$ – Kobe-Wan Kenobi Dec 30 '14 at 9:03

Mahalanobis distances is strongly connected to Gaussian Mixture Modeling with the full covariance mixture model.

When your clusters are from k-means, I would use sum-of-squares (because that is what k-means used).

If your clusters are GMM clusters with a covariance model, I would use these covariance matrixes and Mahalanobis distance. I would also use the cluster weights, maybe just use the full Gaussian density as done by GMM.

In short: it's best to use the same measure that your clustering algorithm used. If you want to use Mahalanobis, use a clustering algorithm which also used Mahalanobis.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.