4
$\begingroup$

I have been doing some simple Mahalanobis distance problems by hand and checking my answers in the back of the book. Things were going well, so I decided to try and write a simple program to calculate it for a real data set using my statistics software. Equation I use is:

$$ d(x,y) = \sqrt{(x-y)^\top \Sigma^{-1}(x-y)} $$

The problem is, after I take the inner product of the terms inside the square root I get a negative value -- at which point my software throws an error. It's not totally clear how to trouble shoot this. My hunch was that I used the wrong representation for the variance covariance matrix (my software has a built-in functionality for variance covariance matrix), but after double checking, it seems to be correct. If I'm not mistaken, the variance covariance matrix can have negative values.

My Question: Does the Mahalanobis formula have a protocol for this? Or is this a sign I did something terribly wrong in my calculations?

$\endgroup$
2
  • $\begingroup$ Please correct the spelling of "Mahalanobis" in your title and in the body text. You have the spelling correct once (though it should be capitalized), and then incorrect both other times. It's particularly important to have it correct in your title. $\endgroup$
    – Glen_b
    Commented Oct 28, 2017 at 7:45
  • $\begingroup$ So if your covariance matrix isn't positive definite, then you cannot use the Mahalanobis distance, correct? $\endgroup$ Commented Mar 5, 2020 at 20:22

2 Answers 2

6
$\begingroup$

For Mahalanobis distance to be a valid distance, $\Sigma$ must be a positive definite matrix. This stems directly from the definition of a positive definite matrix, and the non-negativity axiom of distance. (Whether or not $\Sigma$ has negative entires is not important here; what is important is its eigenvalues.)

So if you're getting negative distances, something has gone wrong. Maybe what you're using for $\Sigma$ is not PD, or, equivalently, what you're using for $\Sigma^{-1}$ is not PD. Or $\Sigma$ is numerically singular. Maybe there's a bug in your code or the software.

$\endgroup$
8
  • $\begingroup$ My data seems pretty run of the mill, I'm not sure why it wouldn't be positive definitive. However on closer inspection I noticed I have a lot of near-zero entries, like 1e-23. I'm not sure how much of a red flag it would be considered. Would flush to zero / floating point play a role? $\endgroup$ Commented Oct 28, 2017 at 3:32
  • 1
    $\begingroup$ What entries are near-zero? Eigenvalues to $\Sigma$? Something else? $\endgroup$
    – Sycorax
    Commented Oct 28, 2017 at 5:22
  • 1
    $\begingroup$ @Sycorax Turns out the negative values were due to a classic case of user error. My mu vector was not being correctly subtracted from the x transpose term while the normal x vector was being correctly demeaned. This resulted in an invalid distance. In hindsight, I should have just took the difference and then transposed. $\endgroup$ Commented Oct 28, 2017 at 7:40
  • 2
    $\begingroup$ @Arash, A covariance or sscp matrix can have negative eigenvalues if there were missing values which were treated by "pairwise" deletion. Also, very tiny negative eigenvalues can appear (in place of zero ones) due to computation precision limits (round-offs) in case the matrix is actually singular. $\endgroup$
    – ttnphns
    Commented Oct 28, 2017 at 12:52
  • 1
    $\begingroup$ @Ben As I write in my answer, it doesn't matter if the elements of the covariance matrix are strictly positive. What does matter is that it's positive definite. $\endgroup$
    – Sycorax
    Commented Feb 26, 2018 at 17:11
-2
$\begingroup$

A common reason you can have the mahalanobi's distance as negative is when your mean difference(miu1 - miu2) have entries with negative signs. You can eliminate this by multiplying the criterion by (-1), which leads to (miu2 - miu1). Remember the inequality for the criterion changes also.

$\endgroup$
1
  • 1
    $\begingroup$ It is hard to follow what you are saying, but it looks incorrect. Any covariance matrix (calculated correctly) will have non-negative eigenvalues; thus, the equation given in the OP cannot be negative. The mean differences can have exclusively negative values in the vectors (e.g. $\mu_1 - \mu_2(-1, -2, -6, -4)$). // What you write appears to be to calculate $d(y, x)$ if $d(x, y)$ gives an impossible value, but $d(x, y) = d(y, x)$. (After all, Mahalanobis distance is a metric.) $\endgroup$
    – Dave
    Commented Jun 29, 2021 at 22:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.