3
$\begingroup$

Assume following: There are ten individuals and each is represented by two properties (Size and Gender). Now I measure the distance between two individuals A and B via the Mahalanobis distance:

$$ d(x, y)=\sqrt{\left(\bar{x}-\bar{y}\right)^{T}S^{-1}\left(\bar{x}-\bar{y}\right)} $$

and get a certain distance. After that I add a few more individuals to the pool and I measure again the distance of the same both individuals A and B as before: Will it be different then (because of the covariance matrix)?

edit: I just tried it out and it's different / the distances between individuals differ with the pool they are part of it.

Improve this question
$\endgroup$
4
  • 3
    $\begingroup$ Your notation is a bit unclear. What do you mean by the, $x$, $y$, $\bar{x}$ and $\bar{y}$? Also, is $S$ a given covariance matrix, or do you estimate it from the sample? What is the multi-dimensional variable in this problem? $\endgroup$
    – Sextus Empiricus
    Commented Dec 22, 2017 at 15:23
  • $\begingroup$ x and y are the individuals (e.g. x is individual A and y is individual B). I estimate it from the sample/pool? Is there a difference between sample and pool? In this example the variable would be A = x = (Size_{A}, Gender_{A}) and B = y = (Size_{B}, Gender_{B}). $\endgroup$
    – Ben
    Commented Dec 22, 2017 at 15:32
  • 1
    $\begingroup$ That is still not very complete. What are $\bar{x}$ and $\bar{y}$? How do you estimate $S$? New questions that arise are: 1 what is the meaning of your distance in the case of categorical variables (gender)? 2 how do you do the multiplication of the different types of variables? 3 What is the meaning of this distance operation, why do you do it? $\endgroup$
    – Sextus Empiricus
    Commented Dec 22, 2017 at 17:44
  • $\begingroup$ x and y are the vectors of the individuals comprising their properties. How I estimate S? Do you mean how I program it? I've done it via stackoverflow.com/a/15142446/6761328 . Relative to your questions: 1) I assume there is no meaning 2) Guess this is connected to the implementation of the calculation I've cited before(?) 3) I want to measure how similar the individuals are with respect to each other. My motivation is to find pairings (of man/woman) which are as similar as possible. $\endgroup$
    – Ben
    Commented Dec 24, 2017 at 9:23

1 Answer 1

Reset to default
5
$\begingroup$

It is only because you have a small sample size and thus a "poor" estimation of $S$, which may be greatly influenced by the new individuals. Try it out with $n < 10000$ and, as long as the new individuals is not a high impact outlier, you should obtain the approximately the same distance between individuals A and B when adding the $10001$th subject.

EDIT: To illustrate the answer below, try this code in R :

> library(MASS)
> S = matrix(c(1, .5,.5,1),2,)
> data = mvrnorm(n=1000,mu=rep(0,2),Sigma=S)
> cov(data)
          [,1]      [,2]
[1,] 1.0301079 0.4881467
[2,] 0.4881467 1.0292714
> round(mahalanobis(data[1:10,],center=mean(data),cov=cov(data)),4)
 [1] 1.1544 2.5189 0.2999 0.6342 2.4010 2.2885 1.2370 1.6227 1.1656
[10] 1.3078
> new = mvrnorm(n=1,mu=rep(0,2),Sigma=S)
> data2= rbind(data,new)
> round(mahalanobis(data2[1:10,],center=mean(data),cov=cov(data)),4)
 [1] 1.1544 2.5189 0.2999 0.6342 2.4010 2.2885 1.2370 1.6227 1.1656
[10] 1.3078

You can see that the 10 persons kept the same distance. We can even change the new $n$ individuals to 100 and mahalanobis distance won't change much

> library(MASS)
> S = matrix(c(1, .5,.5,1),2,)
> data = mvrnorm(n=1000,mu=rep(0,2),Sigma=S)
> cov(data)
          [,1]      [,2]
[1,] 0.9506534 0.4780387
[2,] 0.4780387 0.9835325
> round(mahalanobis(data[1:10,],center=mean(data),cov=cov(data)),4)
 [1] 3.5866 7.0324 1.1189 2.0611 4.5880 0.1649 3.2801 4.4306 2.4925
[10] 1.9930
> new = mvrnorm(n=100,mu=rep(0,2),Sigma=S)
> data2= rbind(data,new)
> round(mahalanobis(data2[1:10,],center=mean(data),cov=cov(data)),4)
 [1] 3.5866 7.0324 1.1189 2.0611 4.5880 0.1649 3.2801 4.4306 2.4925
[10] 1.9930

Considering all participants :

> library(MASS)
> S = matrix(c(1, .5,.5,1),2,)
> data = mvrnorm(n=1000,mu=rep(0,2),Sigma=S)
> cov(data)
          [,1]      [,2]
[1,] 0.9580746 0.4467521
[2,] 0.4467521 0.9838970
> head(round(mahalanobis(data,center=mean(data),cov=cov(data)),4))
[1] 0.3431 2.9275 5.2390 0.1425 0.5218 0.3120
> new = mvrnorm(n=100,mu=rep(0,2),Sigma=S)
> data2= rbind(data,new)
> head(round(mahalanobis(data2,center=mean(data),cov=cov(data)),4))
[1] 0.3431 2.9275 5.2390 0.1425 0.5218 0.3120

Mahalanobis distances remain approximately the same.

Improve this answer
$\endgroup$
4
  • $\begingroup$ But I "only" have about 1000 elements. The difference between the same individuals being part of a pool with 500 and 1000 elements is about 0.2 in my case (absolute it is 0.7 for 500 elements and about 0.9 for 1000 elements). $\endgroup$
    – Ben
    Commented Dec 22, 2017 at 14:42
  • $\begingroup$ Thanks for displaying the situation. Nevertheless, in my case, the difference is prominent. That's probably because of my data pool, there are 30/1000 elements/individuals which are up to 20 standard deviations away from the center. About 800 are within one standard deviation. And there differences for the mahalanobis distance become obvious. $\endgroup$
    – Ben
    Commented Dec 24, 2017 at 9:14
  • $\begingroup$ You can check if your 30 individuals are multivariate outliers (for instance a 5feet tall, 350 pounds guy) or if they are more related to a certain gender? It also depends if the 30 individuals are all in the first or added data. $\endgroup$
    – POC
    Commented Dec 24, 2017 at 18:02
  • $\begingroup$ They are definitely outliers but I need them. Why does the order play a role? $\endgroup$
    – Ben
    Commented Dec 26, 2017 at 14:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.