26
$\begingroup$

If I have a multivariate normal i.i.d. sample $X_1, \ldots, X_n \sim N_p(\mu,\Sigma)$, and define $$d_i^2(b,A) = (X_i - b)' A^{-1} (X_i - b)$$ (which is sort of a Mahalanobis distance [squared] from a sample point to the vector $a$ using the matrix $A$ for weighting), what is the distribution of $d_i^2(\bar X,S)$ (Mahalanobis distance to the sample mean $\bar X$ using the sample covariance matrix $S$)?

I am looking at a paper that claims it is $\chi^2_p$, but this is obviously wrong: the $\chi^2_p$ distribution would have been obtained for $d_i^2(\mu,\Sigma)$ using the (unknown) population mean vector and covariance matrix. When the sample analogues are plugged in, one ought to get a Hotelling $T^{\ 2}$ distribution, or a scaled $F(\cdot)$ distribution, or something like that, but not the $\chi^2_p$. I could not find the exact result either in Muirhead (2005), nor in Anderson (2003), nor in Mardia, Kent and Bibby (1979, 2003) . Apparently, these guys did not bother with outlier diagnostics, as the multivariate normal distribution is perfect and is easily obtained every time one collects multivariate data :-/.

Things may be more complicated than that. The Hotelling $T^{\ 2}$ distribution result is based on assuming independence between the vector part and the matrix part; such independence holds for $\bar X$ and $S$, but it no longer holds for $X_i$ and $S$.

Improve this question
$\endgroup$
9
  • $\begingroup$ In the definition of $d_i^2$, do you still view $X_i$ as a random variable or are you now treating it as a fixed vector? Including the subscript suggests the latter, but that seems a little strange. $\endgroup$
    – whuber ♦
    Commented Jan 3, 2012 at 15:35
  • 1
    $\begingroup$ Just a little off-the-cuff side note, but notice that $X_i - \bar{X}$ is ancillary with respect to $\mu$ and $\sum_i d_i^2(\bar{X},S)$ is equal to a fixed constant (should be $n-p$, or similar, I think) almost surely. $\endgroup$
    – cardinal
    Commented Jan 3, 2012 at 16:00
  • 1
    $\begingroup$ @whuber - perhaps to emphasize that it's calculated using an observation from the sample, not a new observation? $\endgroup$
    – jbowman
    Commented Jan 3, 2012 at 16:57
  • 1
    $\begingroup$ @whuber, roughly along the lines of what jbowman said -- to indicate that this is an observation-level statistic (as opposed to a sample level statistic, like sample mean). $\endgroup$
    – StasK
    Commented Jan 3, 2012 at 17:48
  • 1
    $\begingroup$ The distribution of $d_i^2(\bar X,S)$ is a beta, $n/(n-1)^2 d_i^2(\bar X,S) \sim B(p/2, (n-p-1)/2)$, but I'm still seeking for the distribution of $d^2_i(\mu, S)$. The distributions of the $d^2_i$'s are not independent. $\endgroup$
    – user8331
    Commented Jan 4, 2012 at 14:17

2 Answers 2

Reset to default
23
$\begingroup$

Check out Gaussian Mixture Modeling by Exploiting the Mahalanobis Distance (alternative link). See page no 13, Second column. Authors also given some proof also for deriving the distribution. The distribution is scaled beta. Please let me know if this is not working for you. Otherwise I could check any hint in the S.S. Wilks book tomorrow.

Improve this answer
$\endgroup$
1
  • 4
    $\begingroup$ The answer given in the paper is: $\frac{n}{(n-1)^2} d_i^2(\bar X, S) \sim B(\frac{p}{2}, \frac{n-p-1}{2} )$. Thanks! $\endgroup$
    – StasK
    Commented Jan 3, 2012 at 17:53
13
$\begingroup$

There are 3 relevant distributions. As noted, if the true population parameters are used then the distribution is chi-squared with $df=p$. This is also the asymptotic distribution with estimated parameters and large sample size.

Another answer gives the correct distribution for the most common situation, with estimated parameters when the observation itself is part of the estimation set: $$ \frac{n(d^2)}{(n-1)^2} \sim Beta\left(\frac{p}{2}, \frac{(n-p-1)}{2}\right). $$ However, if the observation $x_i$ is independent of the parameter estimates, then the distribution is proportional to a Fisher's F-ratio distribution: $$ \left(\frac{nd^2(n-p)}{(p(n-1)(n+1)}\right) \sim F(p, n-p) $$

Improve this answer
$\endgroup$
4
  • $\begingroup$ Welcome to the site, @JoeSullivan. I took the liberty of using $\LaTeX$ to make your equations easier to read. Please make sure they still say what you want. $\endgroup$
    – gung - Reinstate Monica
    Commented Jan 27, 2013 at 4:27
  • $\begingroup$ can you give a reference for the F formula? $\endgroup$
    – o17t H1H' S'k
    Commented Jul 2, 2015 at 14:19
  • 1
    $\begingroup$ one related reference, section 3 in Hardin, Johanna, and David M. Rocke. 2005. “The Distribution of Robust Distances.” Journal of Computational and Graphical Statistics 14 (4): 928–46. doi:10.1198/106186005X77685. $\endgroup$
    – Josef
    Commented Nov 12, 2016 at 4:06
  • $\begingroup$ @Josef the version of Hardin et al I found online has the scaling constant $n / (n-1)^2$ inverted! Simple simulations show that the form given here is the correct one. $\endgroup$
    – shabbychef
    Commented Feb 13, 2020 at 16:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.