15
$\begingroup$

In the case of a sample from a one dimensional normal distribution $x \sim \mathcal{N}(\mu, \sigma)$, I can calculate whether or not a sample is within some multiple $\eta$ of $\sigma$ by measuring if x is outside of $\mu \pm \eta\sigma$.

How about the case of a MVN? How can I make the same measurement? Specifically, given...

$$ \begin{bmatrix}x_1 \\ x_2\end{bmatrix} \sim \mathcal{N}\Big( \begin{bmatrix}\mu_1 \\ \mu_2\end{bmatrix}, \begin{bmatrix}\sigma_{11} & \sigma_{12} \\ \sigma_{21} & \sigma_{22}\end{bmatrix} \Big) $$

How can I calculate if $\mathbf{x}$ is within $\pm\eta\Sigma$?

$\endgroup$

1 Answer 1

11
$\begingroup$

The usual metric to use here is the scaled Mahalanobis distance:

$$S(\mathbf{x}) \equiv \frac{D(\mathbf{x})}{\sqrt{n}} = \sqrt{\frac{(\mathbf{x} - \boldsymbol{\mu})^\text{T} \mathbf{\Sigma}^{-1} (\mathbf{x} - \boldsymbol{\mu})}{n}}.$$

If we let $\mathbf{\Sigma}^{1/2}$ denote the multivariate "standard deviation matrix" (i.e., the principal square root of the variance matrix) then this distance can be written in alternative form as:

$$S(\mathbf{x}) = \frac{||\mathbf{\Sigma}^{-1/2} (\mathbf{x} - \boldsymbol{\mu})||}{\sqrt{n}},$$

which is a scaled version of the norm of the standardised vector (with scaling adjusting to remove the effect on the norm from the number of elements in the vector). This measure has a few useful properties that make it a good measure of standardised distance from the mean. In particular, in the special case where $n=1$ you get $S(\mathbf{x}) = |x - \mu|/\sigma$, which is the absolute standardised distance from the mean. (Consequently, you get $x = \mu \pm S(\mathbf{x}) \sigma$ in this special case.) The broader distance measure generalises the univariate notion of the standardised distance from the sample mean in a way that accounts for the variance and covariance in the variance matrix, and the length of the random vector.

Using the scaled Mahalanobis distance you can compute whether or not the value $\mathbf{x}$ lies within an elliptical region centred on the mean vector within a particular scaled distance $r$. This region can be determined as:

$$\begin{align} \mathscr{X}(r) &\equiv \{ \mathbf{x} \in \mathbb{R}^n | S(\mathbf{x}) \leqslant r \} \\[6pt] &= \{ \mathbf{x} \in \mathbb{R}^n | S(\mathbf{x})^2 \leqslant r^2 \} \\[6pt] &= \{ \mathbf{x} \in \mathbb{R}^n | (\mathbf{x} - \boldsymbol{\mu})^\text{T} \mathbf{\Sigma}^{-1} (\mathbf{x} - \boldsymbol{\mu}) \leqslant n r^2 \}. \\[6pt] \end{align}$$

In the case of normal data you have $S(\mathbf{X}) \sim \text{Chi}(n)/\sqrt{n}$ using the chi distribution, so it is possible to compute the probability of lying in one of these regions for a specific value of $r$. Specifically, you get:

$$\begin{align} \mathbb{P}(X \in \mathscr{X}(r)) &= \mathbb{P}(S(\mathbf{X}) \leqslant r) \\[16pt] &= \int \limits_0^{r} \text{Chi}(s|n) \ ds \\[6pt] &= \frac{2 (n/2)^{n/2}}{\Gamma(n/2)} \int \limits_0^{r} s^{n-1} \exp \Big( - \frac{n s^2}{2} \Big) \ ds. \\[6pt] \end{align}$$

$\endgroup$
10
  • $\begingroup$ hmm ok. Then does a scaled Malahanobis distance of 1, 2, 3, ... correspond directly to standard deviations or have I missed something? $\endgroup$
    – Joff
    Apr 22, 2021 at 6:07
  • 2
    $\begingroup$ As you can see from the special case where $n=1$, the scaled Mahalanobis distance corresponds directly to distances in standard deviations in that case. Consequently, the more general distance measure generalises this univariate measure. $\endgroup$
    – Ben
    Apr 22, 2021 at 6:09
  • 1
    $\begingroup$ You have not previously mentioned any confidence intervals or assumptions about same, so that is not possible for me to answer. $\endgroup$
    – Ben
    Apr 22, 2021 at 6:12
  • 1
    $\begingroup$ That no longer holds for $n>1$. The relevant probability coverage can be obtained by using the chi distribution (or chi-squared distribution) using the above results. $\endgroup$
    – Ben
    Apr 22, 2021 at 6:23
  • 1
    $\begingroup$ If I'm not mistaken, $S(\mathbf{X}) \sim \text{Chi}(n)/\sqrt{n}$ follows a Nakagami$(n/2, 1)$-distribution (shape/spread-parametrization). $n$ being the dimension of the normal distribution. $\endgroup$ Apr 22, 2021 at 8:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy