23
$\begingroup$

If I have a multivariate normal i.i.d. sample $X_1, \ldots, X_n \sim N_p(\mu,\Sigma)$, and define $$d_i^2(b,A) = (X_i - b)' A^{-1} (X_i - b)$$ (which is sort of a Mahalanobis distance [squared] from a sample point to the vector $a$ using the matrix $A$ for weighting), what is the distribution of $d_i^2(\bar X,S)$ (Mahalanobis distance to the sample mean $\bar X$ using the sample covariance matrix $S$)?

I am looking at a paper that claims it is $\chi^2_p$, but this is obviously wrong: the $\chi^2_p$ distribution would have been obtained for $d_i^2(\mu,\Sigma)$ using the (unknown) population mean vector and covariance matrix. When the sample analogues are plugged in, one ought to get a Hotelling $T^{\ 2}$ distribution, or a scaled $F(\cdot)$ distribution, or something like that, but not the $\chi^2_p$. I could not find the exact result either in Muirhead (2005), nor in Anderson (2003), nor in Mardia, Kent and Bibby (1979, 2003) . Apparently, these guys did not bother with outlier diagnostics, as the multivariate normal distribution is perfect and is easily obtained every time one collects multivariate data :-/.

Things may be more complicated than that. The Hotelling $T^{\ 2}$ distribution result is based on assuming independence between the vector part and the matrix part; such independence holds for $\bar X$ and $S$, but it no longer holds for $X_i$ and $S$.

$\endgroup$
  • $\begingroup$ In the definition of $d_i^2$, do you still view $X_i$ as a random variable or are you now treating it as a fixed vector? Including the subscript suggests the latter, but that seems a little strange. $\endgroup$ – whuber Jan 3 '12 at 15:35
  • 1
    $\begingroup$ Just a little off-the-cuff side note, but notice that $X_i - \bar{X}$ is ancillary with respect to $\mu$ and $\sum_i d_i^2(\bar{X},S)$ is equal to a fixed constant (should be $n-p$, or similar, I think) almost surely. $\endgroup$ – cardinal Jan 3 '12 at 16:00
  • 1
    $\begingroup$ @whuber - perhaps to emphasize that it's calculated using an observation from the sample, not a new observation? $\endgroup$ – jbowman Jan 3 '12 at 16:57
  • 1
    $\begingroup$ @whuber, roughly along the lines of what jbowman said -- to indicate that this is an observation-level statistic (as opposed to a sample level statistic, like sample mean). $\endgroup$ – StasK Jan 3 '12 at 17:48
  • 1
    $\begingroup$ The distribution of $d_i^2(\bar X,S)$ is a beta, $n/(n-1)^2 d_i^2(\bar X,S) \sim B(p/2, (n-p-1)/2)$, but I'm still seeking for the distribution of $d^2_i(\mu, S)$. The distributions of the $d^2_i$'s are not independent. $\endgroup$ – user8331 Jan 4 '12 at 14:17
18
$\begingroup$

Check out Gaussian Mixture Modeling by Exploiting the Mahalanobis Distance (alternative link). See page no 13, Second column. Authors also given some proof also for deriving the distribution. The distribution is scaled beta. Please let me know if this is not working for you. Otherwise I could check any hint in the S.S. Wilks book tomorrow.

$\endgroup$
  • 4
    $\begingroup$ The answer given in the paper is: $\frac{n}{(n-1)^2} d_i^2(\bar X, S) \sim B(\frac{p}{2}, \frac{n-p-1}{2} )$. Thanks! $\endgroup$ – StasK Jan 3 '12 at 17:53
9
$\begingroup$

There are 3 relevant distributions. As noted, if the true population parameters are used then the distribution is chi-squared with $df=p$. This is also the asymptotic distribution with estimated parameters and large sample size.

Another answer gives the correct distribution for the most common situation, with estimated parameters when the observation itself is part of the estimation set: $$ \frac{n(d^2)}{(n-1)^2} \sim Beta\left(\frac{p}{2}, \frac{(n-p-1)}{2}\right). $$ However, if the observation $x_i$ is independent of the parameter estimates, then the distribution is proportional to a Fisher's F-ratio distribution: $$ \left(\frac{nd^2(n-p)}{(p(n-1)(n+1)}\right) \sim F(p, n-p) $$

$\endgroup$
  • $\begingroup$ Welcome to the site, @JoeSullivan. I took the liberty of using $\LaTeX$ to make your equations easier to read. Please make sure they still say what you want. $\endgroup$ – gung - Reinstate Monica Jan 27 '13 at 4:27
  • $\begingroup$ can you give a reference for the F formula? $\endgroup$ – eyaler Jul 2 '15 at 14:19
  • 1
    $\begingroup$ one related reference, section 3 in Hardin, Johanna, and David M. Rocke. 2005. “The Distribution of Robust Distances.” Journal of Computational and Graphical Statistics 14 (4): 928–46. doi:10.1198/106186005X77685. $\endgroup$ – Josef Nov 12 '16 at 4:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.