7
$\begingroup$

The Wikipedia entry on Mahalanobis Distance contains this note:

Another intuitive description of Mahalanobis distance is that it is square root of the negative log likelihood. That is, the exponential of the negative square of the Mahalanobis distance will give you the likelihood of your data point belonging to (a presumed normal) distribution of the sample points you already have.

How/where is this shown? What does it mean (I guess it's not so intuitive...)?

$\endgroup$
  • 1
    $\begingroup$ This is the topic from Bayes classification. You can find a number of good answers here covering it. But let me mention this time just one of mine. The "likelihood" of a class given the data point, k|x, there is expressed pdf function, where the exponent of the squared Mahalanobis d you may notice in the numerator. $\endgroup$ – ttnphns May 12 '14 at 18:01
  • 2
    $\begingroup$ The quotation is only partly correct: the log likelihood is given by negative one-half the square root of the Mahalanobis distance plus one-half the log of the determinant of the inverse covariance matrix, $\det(S^{-1})$, minus $(n/2)\log(2\pi)$ (in $n$ dimensions). For the purpose of comparing likelihoods with the same covariance matrix the additive factors can be dropped, but the initial factor of $1/2$ is crucial for implementing and interpreting certain tests. $\endgroup$ – whuber May 12 '14 at 18:17
  • $\begingroup$ @whuber, ttnphns: Can you write out the exact equations as an answer? What does it mean? $\endgroup$ – Adi Shavit May 13 '14 at 9:06
  • $\begingroup$ @whuber: I just found this equation on Wikipedia: en.wikipedia.org/wiki/…. It is a bit different from your comment. Which is correct? $\endgroup$ – Adi Shavit May 13 '14 at 19:15
  • 1
    $\begingroup$ How do the two differ? It looks the same to me. :-) It might help to know that $\Sigma=S^{-1}$, which flips the sign of the logarithm of the determinant in the formula. $\endgroup$ – whuber May 13 '14 at 19:18
6
$\begingroup$

For completeness, I'll answer myself.
I eventually found the equation here, and as @whuber writes in his comment:

Define:

  • $k$: the multivariate dimension
  • $\mu$: the multivariate mean (a $k$-dimensional vector);
  • $\Sigma$: the $k\times k$ covariance matrix;

Then:

  • The squared Mahalanobis Distance is: $D^2=(x-\mu)^T\Sigma^{-1}(x-\mu)$
  • The log-likelihood is: $ln(L)=-\frac12ln(|\Sigma|)-\frac12(x-\mu)^T\Sigma^{-1}(x-\mu)-\frac k2 ln(2\pi)$
    or: $ln(L)=-\frac12ln(|\Sigma|)-\frac12D^2-\frac k2 ln(2\pi)$

Thus, define $c\equiv\frac12(ln(|\Sigma|) + k\cdot ln(2\pi))$ then:

  • $ln(L)=-\frac12D^2-c$ and;
  • $D=\sqrt{-2(ln(L)+c)} $
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.