Skip to main content
Cross Validated

Return to Answer

added 28 characters in body
Source Link
Ben
Ben
  • 132.9k
  • 7
  • 255
  • 588

The answer by Macro is great, but here is an even simpler way that does not require you to use any outside theorem asserting the conditional distribution. It involves writing the Mahalanobis distance in a form that separates the argument variable for the conditioning statement, and then factorising the normal density accordingly.

Rewriting the Mahalanobis distance for a conditional vector: This derivation uses a matrix inversion formula that uses the Schur complement $\boldsymbol{\Sigma}_* \equiv \boldsymbol{\Sigma}_{11} - \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21}$. We first use the blockwise inversion formula to write the inverse-variance matrix as:

$$\begin{equation} \begin{aligned} \boldsymbol{\Sigma}^{-1} = \begin{bmatrix} \boldsymbol{\Sigma}_{11} & \boldsymbol{\Sigma}_{12} \\ \boldsymbol{\Sigma}_{21} & \boldsymbol{\Sigma}_{22} \\ \end{bmatrix}^{-1} = \begin{bmatrix} \boldsymbol{\Sigma}_{11}^* & \boldsymbol{\Sigma}_{12}^* \\ \boldsymbol{\Sigma}_{21}^* & \boldsymbol{\Sigma}_{22}^* \\ \end{bmatrix}, \end{aligned} \end{equation}$$

where:

$$\begin{equation} \begin{aligned} \begin{matrix} \boldsymbol{\Sigma}_{11}^* = \boldsymbol{\Sigma}_*^{-1} \text{ } \quad \quad \quad \quad & & & & & \boldsymbol{\Sigma}_{12}^* = -\boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1}, \quad \quad \quad \\[6pt] \boldsymbol{\Sigma}_{21}^* = - \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} & & & & & \boldsymbol{\Sigma}_{22}^* = \boldsymbol{\Sigma}_{22}^{-1} + \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1}. \text{ } \\[6pt] \end{matrix} \end{aligned} \end{equation}$$

Using this formula we can now write the Mahalanobis distance as:

$$\begin{equation} \begin{aligned} (\boldsymbol{y} &- \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) \\[6pt] &= \begin{bmatrix} \boldsymbol{y}_1 - \boldsymbol{\mu}_1 \\ \boldsymbol{y}_2 - \boldsymbol{\mu}_2 \end{bmatrix}^\text{T} \begin{bmatrix} \boldsymbol{\Sigma}_{11}^* & \boldsymbol{\Sigma}_{12}^* \\ \boldsymbol{\Sigma}_{21}^* & \boldsymbol{\Sigma}_{22}^* \\ \end{bmatrix} \begin{bmatrix} \boldsymbol{y}_1 - \boldsymbol{\mu}_1 \\ \boldsymbol{y}_2 - \boldsymbol{\mu}_2 \end{bmatrix} \\[6pt] &= \quad (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_{11}^* (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_{12}^* (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{21}^* (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^* (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= \quad (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) - (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad - (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= (\boldsymbol{y}_1 - (\boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)))^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - (\boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2))) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) , \\[6pt] \end{aligned} \end{equation}$$

where $\boldsymbol{\mu}_* \equiv \boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)$ is the conditional mean vector(which will be the conditional mean vector in the normal case). Note that this result is a general result that does not assume normality of the random vectors involved in the decomposition. It gives a useful way of decomposing the Mahalanobis distance so that it consists of a sum of quadratic forms on the marginal and conditional parts. In the conditional part the conditioning vector $\boldsymbol{y}_2$ is absorbed into the mean vector and variance matrix. To clarify the form, we repeat the equation with labelling of terms:

$$(\boldsymbol{y} - \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) = \underbrace{(\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)}_\text{Conditional Part} + \underbrace{(\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)}_\text{Marginal Part}.$$

Deriving the conditional distribution: Now that we have the above form for the Mahalanobis distance, the rest is easy. We have:

$$\begin{equation} \begin{aligned} p(\boldsymbol{y}_1 | \boldsymbol{y}_2, \boldsymbol{\mu}, \boldsymbol{\Sigma}) &\overset{\boldsymbol{y}_1}{\propto} p(\boldsymbol{y}_1 , \boldsymbol{y}_2 | \boldsymbol{\mu}, \boldsymbol{\Sigma}) \\[12pt] &= \text{N}(\boldsymbol{y} | \boldsymbol{\mu}, \boldsymbol{\Sigma}) \\[10pt] &\overset{\boldsymbol{y}_1}{\propto} \exp \Big( - \frac{1}{2} (\boldsymbol{y} - \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) \Big) \\[6pt] &\overset{\boldsymbol{y}_1}{\propto} \exp \Big( - \frac{1}{2} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*) \Big) \\[6pt] &\overset{\boldsymbol{y}_1}{\propto}\text{N}(\boldsymbol{y}_1 | \boldsymbol{\mu}_*, \boldsymbol{\Sigma}_*). \\[6pt] \end{aligned} \end{equation}$$

This establishes that the conditional distribution is also multivariate normal, with the specified conditional mean vector and conditional variance matrix.

The answer by Macro is great, but here is an even simpler way that does not require you to use any outside theorem asserting the conditional distribution. It involves writing the Mahalanobis distance in a form that separates the argument variable for the conditioning statement, and then factorising the normal density accordingly.

Rewriting the Mahalanobis distance for a conditional vector: This derivation uses a matrix inversion formula that uses the Schur complement $\boldsymbol{\Sigma}_* \equiv \boldsymbol{\Sigma}_{11} - \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21}$. We first use the blockwise inversion formula to write the inverse-variance matrix as:

$$\begin{equation} \begin{aligned} \boldsymbol{\Sigma}^{-1} = \begin{bmatrix} \boldsymbol{\Sigma}_{11} & \boldsymbol{\Sigma}_{12} \\ \boldsymbol{\Sigma}_{21} & \boldsymbol{\Sigma}_{22} \\ \end{bmatrix}^{-1} = \begin{bmatrix} \boldsymbol{\Sigma}_{11}^* & \boldsymbol{\Sigma}_{12}^* \\ \boldsymbol{\Sigma}_{21}^* & \boldsymbol{\Sigma}_{22}^* \\ \end{bmatrix}, \end{aligned} \end{equation}$$

where:

$$\begin{equation} \begin{aligned} \begin{matrix} \boldsymbol{\Sigma}_{11}^* = \boldsymbol{\Sigma}_*^{-1} \text{ } \quad \quad \quad \quad & & & & & \boldsymbol{\Sigma}_{12}^* = -\boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1}, \quad \quad \quad \\[6pt] \boldsymbol{\Sigma}_{21}^* = - \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} & & & & & \boldsymbol{\Sigma}_{22}^* = \boldsymbol{\Sigma}_{22}^{-1} + \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1}. \text{ } \\[6pt] \end{matrix} \end{aligned} \end{equation}$$

Using this formula we can now write the Mahalanobis distance as:

$$\begin{equation} \begin{aligned} (\boldsymbol{y} &- \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) \\[6pt] &= \begin{bmatrix} \boldsymbol{y}_1 - \boldsymbol{\mu}_1 \\ \boldsymbol{y}_2 - \boldsymbol{\mu}_2 \end{bmatrix}^\text{T} \begin{bmatrix} \boldsymbol{\Sigma}_{11}^* & \boldsymbol{\Sigma}_{12}^* \\ \boldsymbol{\Sigma}_{21}^* & \boldsymbol{\Sigma}_{22}^* \\ \end{bmatrix} \begin{bmatrix} \boldsymbol{y}_1 - \boldsymbol{\mu}_1 \\ \boldsymbol{y}_2 - \boldsymbol{\mu}_2 \end{bmatrix} \\[6pt] &= \quad (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_{11}^* (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_{12}^* (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{21}^* (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^* (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= \quad (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) - (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad - (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= (\boldsymbol{y}_1 - (\boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)))^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - (\boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2))) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) , \\[6pt] \end{aligned} \end{equation}$$

where $\boldsymbol{\mu}_* \equiv \boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)$ is the conditional mean vector. Note that this result is a general result that does not assume normality of the random vectors involved in the decomposition. It gives a useful way of decomposing the Mahalanobis distance so that it consists of a sum of quadratic forms on the marginal and conditional parts. In the conditional part the conditioning vector $\boldsymbol{y}_2$ is absorbed into the mean vector and variance matrix. To clarify the form, we repeat the equation with labelling of terms:

$$(\boldsymbol{y} - \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) = \underbrace{(\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)}_\text{Conditional Part} + \underbrace{(\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)}_\text{Marginal Part}.$$

Deriving the conditional distribution: Now that we have the above form for the Mahalanobis distance, the rest is easy. We have:

$$\begin{equation} \begin{aligned} p(\boldsymbol{y}_1 | \boldsymbol{y}_2, \boldsymbol{\mu}, \boldsymbol{\Sigma}) &\overset{\boldsymbol{y}_1}{\propto} p(\boldsymbol{y}_1 , \boldsymbol{y}_2 | \boldsymbol{\mu}, \boldsymbol{\Sigma}) \\[12pt] &= \text{N}(\boldsymbol{y} | \boldsymbol{\mu}, \boldsymbol{\Sigma}) \\[10pt] &\overset{\boldsymbol{y}_1}{\propto} \exp \Big( - \frac{1}{2} (\boldsymbol{y} - \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) \Big) \\[6pt] &\overset{\boldsymbol{y}_1}{\propto} \exp \Big( - \frac{1}{2} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*) \Big) \\[6pt] &\overset{\boldsymbol{y}_1}{\propto}\text{N}(\boldsymbol{y}_1 | \boldsymbol{\mu}_*, \boldsymbol{\Sigma}_*). \\[6pt] \end{aligned} \end{equation}$$

This establishes that the conditional distribution is also multivariate normal, with the specified conditional mean vector and conditional variance matrix.

The answer by Macro is great, but here is an even simpler way that does not require you to use any outside theorem asserting the conditional distribution. It involves writing the Mahalanobis distance in a form that separates the argument variable for the conditioning statement, and then factorising the normal density accordingly.

Rewriting the Mahalanobis distance for a conditional vector: This derivation uses a matrix inversion formula that uses the Schur complement $\boldsymbol{\Sigma}_* \equiv \boldsymbol{\Sigma}_{11} - \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21}$. We first use the blockwise inversion formula to write the inverse-variance matrix as:

$$\begin{equation} \begin{aligned} \boldsymbol{\Sigma}^{-1} = \begin{bmatrix} \boldsymbol{\Sigma}_{11} & \boldsymbol{\Sigma}_{12} \\ \boldsymbol{\Sigma}_{21} & \boldsymbol{\Sigma}_{22} \\ \end{bmatrix}^{-1} = \begin{bmatrix} \boldsymbol{\Sigma}_{11}^* & \boldsymbol{\Sigma}_{12}^* \\ \boldsymbol{\Sigma}_{21}^* & \boldsymbol{\Sigma}_{22}^* \\ \end{bmatrix}, \end{aligned} \end{equation}$$

where:

$$\begin{equation} \begin{aligned} \begin{matrix} \boldsymbol{\Sigma}_{11}^* = \boldsymbol{\Sigma}_*^{-1} \text{ } \quad \quad \quad \quad & & & & & \boldsymbol{\Sigma}_{12}^* = -\boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1}, \quad \quad \quad \\[6pt] \boldsymbol{\Sigma}_{21}^* = - \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} & & & & & \boldsymbol{\Sigma}_{22}^* = \boldsymbol{\Sigma}_{22}^{-1} + \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1}. \text{ } \\[6pt] \end{matrix} \end{aligned} \end{equation}$$

Using this formula we can now write the Mahalanobis distance as:

$$\begin{equation} \begin{aligned} (\boldsymbol{y} &- \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) \\[6pt] &= \begin{bmatrix} \boldsymbol{y}_1 - \boldsymbol{\mu}_1 \\ \boldsymbol{y}_2 - \boldsymbol{\mu}_2 \end{bmatrix}^\text{T} \begin{bmatrix} \boldsymbol{\Sigma}_{11}^* & \boldsymbol{\Sigma}_{12}^* \\ \boldsymbol{\Sigma}_{21}^* & \boldsymbol{\Sigma}_{22}^* \\ \end{bmatrix} \begin{bmatrix} \boldsymbol{y}_1 - \boldsymbol{\mu}_1 \\ \boldsymbol{y}_2 - \boldsymbol{\mu}_2 \end{bmatrix} \\[6pt] &= \quad (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_{11}^* (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_{12}^* (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{21}^* (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^* (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= \quad (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) - (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad - (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= (\boldsymbol{y}_1 - (\boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)))^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - (\boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2))) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) , \\[6pt] \end{aligned} \end{equation}$$

where $\boldsymbol{\mu}_* \equiv \boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)$ (which will be the conditional mean vector in the normal case). Note that this result is a general result that does not assume normality of the random vectors involved in the decomposition. It gives a useful way of decomposing the Mahalanobis distance so that it consists of a sum of quadratic forms on the marginal and conditional parts. In the conditional part the conditioning vector $\boldsymbol{y}_2$ is absorbed into the mean vector and variance matrix. To clarify the form, we repeat the equation with labelling of terms:

$$(\boldsymbol{y} - \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) = \underbrace{(\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)}_\text{Conditional Part} + \underbrace{(\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)}_\text{Marginal Part}.$$

Deriving the conditional distribution: Now that we have the above form for the Mahalanobis distance, the rest is easy. We have:

$$\begin{equation} \begin{aligned} p(\boldsymbol{y}_1 | \boldsymbol{y}_2, \boldsymbol{\mu}, \boldsymbol{\Sigma}) &\overset{\boldsymbol{y}_1}{\propto} p(\boldsymbol{y}_1 , \boldsymbol{y}_2 | \boldsymbol{\mu}, \boldsymbol{\Sigma}) \\[12pt] &= \text{N}(\boldsymbol{y} | \boldsymbol{\mu}, \boldsymbol{\Sigma}) \\[10pt] &\overset{\boldsymbol{y}_1}{\propto} \exp \Big( - \frac{1}{2} (\boldsymbol{y} - \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) \Big) \\[6pt] &\overset{\boldsymbol{y}_1}{\propto} \exp \Big( - \frac{1}{2} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*) \Big) \\[6pt] &\overset{\boldsymbol{y}_1}{\propto}\text{N}(\boldsymbol{y}_1 | \boldsymbol{\mu}_*, \boldsymbol{\Sigma}_*). \\[6pt] \end{aligned} \end{equation}$$

This establishes that the conditional distribution is also multivariate normal, with the specified conditional mean vector and conditional variance matrix.

Corrected spelling of Mahalanobis
Source Link
Ben
Ben
  • 132.9k
  • 7
  • 255
  • 588

The answer by Macro is great, but here is an even simpler way that does not require you to use any outside theorem asserting the conditional distribution. It involves writing the MahanalobisMahalanobis distance in a form that separates the argument variable for the conditioning statement, and then factorising the normal density accordingly.

Rewriting the MahanalobisMahalanobis distance for a conditional vector: This derivation uses a matrix inversion formula that uses the Schur complement $\boldsymbol{\Sigma}_* \equiv \boldsymbol{\Sigma}_{11} - \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21}$. We first use the blockwise inversion formula to write the inverse-variance matrix as:

$$\begin{equation} \begin{aligned} \boldsymbol{\Sigma}^{-1} = \begin{bmatrix} \boldsymbol{\Sigma}_{11} & \boldsymbol{\Sigma}_{12} \\ \boldsymbol{\Sigma}_{21} & \boldsymbol{\Sigma}_{22} \\ \end{bmatrix}^{-1} = \begin{bmatrix} \boldsymbol{\Sigma}_{11}^* & \boldsymbol{\Sigma}_{12}^* \\ \boldsymbol{\Sigma}_{21}^* & \boldsymbol{\Sigma}_{22}^* \\ \end{bmatrix}, \end{aligned} \end{equation}$$

where:

$$\begin{equation} \begin{aligned} \begin{matrix} \boldsymbol{\Sigma}_{11}^* = \boldsymbol{\Sigma}_*^{-1} \text{ } \quad \quad \quad \quad & & & & & \boldsymbol{\Sigma}_{12}^* = -\boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1}, \quad \quad \quad \\[6pt] \boldsymbol{\Sigma}_{21}^* = - \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} & & & & & \boldsymbol{\Sigma}_{22}^* = \boldsymbol{\Sigma}_{22}^{-1} + \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1}. \text{ } \\[6pt] \end{matrix} \end{aligned} \end{equation}$$

Using this formula we can now write the MahanalobisMahalanobis distance as:

$$\begin{equation} \begin{aligned} (\boldsymbol{y} &- \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) \\[6pt] &= \begin{bmatrix} \boldsymbol{y}_1 - \boldsymbol{\mu}_1 \\ \boldsymbol{y}_2 - \boldsymbol{\mu}_2 \end{bmatrix}^\text{T} \begin{bmatrix} \boldsymbol{\Sigma}_{11}^* & \boldsymbol{\Sigma}_{12}^* \\ \boldsymbol{\Sigma}_{21}^* & \boldsymbol{\Sigma}_{22}^* \\ \end{bmatrix} \begin{bmatrix} \boldsymbol{y}_1 - \boldsymbol{\mu}_1 \\ \boldsymbol{y}_2 - \boldsymbol{\mu}_2 \end{bmatrix} \\[6pt] &= \quad (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_{11}^* (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_{12}^* (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{21}^* (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^* (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= \quad (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) - (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad - (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= (\boldsymbol{y}_1 - (\boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)))^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - (\boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2))) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) , \\[6pt] \end{aligned} \end{equation}$$

where $\boldsymbol{\mu}_* \equiv \boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)$ is the conditional mean vector. Note that this result is a general result that does not assume normality of the random vectors involved in the decomposition. It gives a useful way of decomposing the MahanalobisMahalanobis distance so that it consists of a sum of quadratic forms on the marginal and conditional parts. In the conditional part the conditioning vector $\boldsymbol{y}_2$ is absorbed into the mean vector and variance matrix. To clarify the form, we repeat the equation with labelling of terms:

$$(\boldsymbol{y} - \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) = \underbrace{(\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)}_\text{Conditional Part} + \underbrace{(\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)}_\text{Marginal Part}.$$

Deriving the conditional distribution: Now that we have the above form for the MahanalobisMahalanobis distance, the rest is easy. We have:

$$\begin{equation} \begin{aligned} p(\boldsymbol{y}_1 | \boldsymbol{y}_2, \boldsymbol{\mu}, \boldsymbol{\Sigma}) &\overset{\boldsymbol{y}_1}{\propto} p(\boldsymbol{y}_1 , \boldsymbol{y}_2 | \boldsymbol{\mu}, \boldsymbol{\Sigma}) \\[12pt] &= \text{N}(\boldsymbol{y} | \boldsymbol{\mu}, \boldsymbol{\Sigma}) \\[10pt] &\overset{\boldsymbol{y}_1}{\propto} \exp \Big( - \frac{1}{2} (\boldsymbol{y} - \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) \Big) \\[6pt] &\overset{\boldsymbol{y}_1}{\propto} \exp \Big( - \frac{1}{2} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*) \Big) \\[6pt] &\overset{\boldsymbol{y}_1}{\propto}\text{N}(\boldsymbol{y}_1 | \boldsymbol{\mu}_*, \boldsymbol{\Sigma}_*). \\[6pt] \end{aligned} \end{equation}$$

This establishes that the conditional distribution is also multivariate normal, with the specified conditional mean vector and conditional variance matrix.

The answer by Macro is great, but here is an even simpler way that does not require you to use any outside theorem asserting the conditional distribution. It involves writing the Mahanalobis distance in a form that separates the argument variable for the conditioning statement, and then factorising the normal density accordingly.

Rewriting the Mahanalobis distance for a conditional vector: This derivation uses a matrix inversion formula that uses the Schur complement $\boldsymbol{\Sigma}_* \equiv \boldsymbol{\Sigma}_{11} - \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21}$. We first use the blockwise inversion formula to write the inverse-variance matrix as:

$$\begin{equation} \begin{aligned} \boldsymbol{\Sigma}^{-1} = \begin{bmatrix} \boldsymbol{\Sigma}_{11} & \boldsymbol{\Sigma}_{12} \\ \boldsymbol{\Sigma}_{21} & \boldsymbol{\Sigma}_{22} \\ \end{bmatrix}^{-1} = \begin{bmatrix} \boldsymbol{\Sigma}_{11}^* & \boldsymbol{\Sigma}_{12}^* \\ \boldsymbol{\Sigma}_{21}^* & \boldsymbol{\Sigma}_{22}^* \\ \end{bmatrix}, \end{aligned} \end{equation}$$

where:

$$\begin{equation} \begin{aligned} \begin{matrix} \boldsymbol{\Sigma}_{11}^* = \boldsymbol{\Sigma}_*^{-1} \text{ } \quad \quad \quad \quad & & & & & \boldsymbol{\Sigma}_{12}^* = -\boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1}, \quad \quad \quad \\[6pt] \boldsymbol{\Sigma}_{21}^* = - \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} & & & & & \boldsymbol{\Sigma}_{22}^* = \boldsymbol{\Sigma}_{22}^{-1} + \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1}. \text{ } \\[6pt] \end{matrix} \end{aligned} \end{equation}$$

Using this formula we can now write the Mahanalobis distance as:

$$\begin{equation} \begin{aligned} (\boldsymbol{y} &- \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) \\[6pt] &= \begin{bmatrix} \boldsymbol{y}_1 - \boldsymbol{\mu}_1 \\ \boldsymbol{y}_2 - \boldsymbol{\mu}_2 \end{bmatrix}^\text{T} \begin{bmatrix} \boldsymbol{\Sigma}_{11}^* & \boldsymbol{\Sigma}_{12}^* \\ \boldsymbol{\Sigma}_{21}^* & \boldsymbol{\Sigma}_{22}^* \\ \end{bmatrix} \begin{bmatrix} \boldsymbol{y}_1 - \boldsymbol{\mu}_1 \\ \boldsymbol{y}_2 - \boldsymbol{\mu}_2 \end{bmatrix} \\[6pt] &= \quad (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_{11}^* (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_{12}^* (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{21}^* (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^* (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= \quad (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) - (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad - (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= (\boldsymbol{y}_1 - (\boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)))^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - (\boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2))) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) , \\[6pt] \end{aligned} \end{equation}$$

where $\boldsymbol{\mu}_* \equiv \boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)$ is the conditional mean vector. Note that this result is a general result that does not assume normality of the random vectors involved in the decomposition. It gives a useful way of decomposing the Mahanalobis distance so that it consists of a sum of quadratic forms on the marginal and conditional parts. In the conditional part the conditioning vector $\boldsymbol{y}_2$ is absorbed into the mean vector and variance matrix. To clarify the form, we repeat the equation with labelling of terms:

$$(\boldsymbol{y} - \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) = \underbrace{(\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)}_\text{Conditional Part} + \underbrace{(\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)}_\text{Marginal Part}.$$

Deriving the conditional distribution: Now that we have the above form for the Mahanalobis distance, the rest is easy. We have:

$$\begin{equation} \begin{aligned} p(\boldsymbol{y}_1 | \boldsymbol{y}_2, \boldsymbol{\mu}, \boldsymbol{\Sigma}) &\overset{\boldsymbol{y}_1}{\propto} p(\boldsymbol{y}_1 , \boldsymbol{y}_2 | \boldsymbol{\mu}, \boldsymbol{\Sigma}) \\[12pt] &= \text{N}(\boldsymbol{y} | \boldsymbol{\mu}, \boldsymbol{\Sigma}) \\[10pt] &\overset{\boldsymbol{y}_1}{\propto} \exp \Big( - \frac{1}{2} (\boldsymbol{y} - \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) \Big) \\[6pt] &\overset{\boldsymbol{y}_1}{\propto} \exp \Big( - \frac{1}{2} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*) \Big) \\[6pt] &\overset{\boldsymbol{y}_1}{\propto}\text{N}(\boldsymbol{y}_1 | \boldsymbol{\mu}_*, \boldsymbol{\Sigma}_*). \\[6pt] \end{aligned} \end{equation}$$

This establishes that the conditional distribution is also multivariate normal, with the specified conditional mean vector and conditional variance matrix.

The answer by Macro is great, but here is an even simpler way that does not require you to use any outside theorem asserting the conditional distribution. It involves writing the Mahalanobis distance in a form that separates the argument variable for the conditioning statement, and then factorising the normal density accordingly.

Rewriting the Mahalanobis distance for a conditional vector: This derivation uses a matrix inversion formula that uses the Schur complement $\boldsymbol{\Sigma}_* \equiv \boldsymbol{\Sigma}_{11} - \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21}$. We first use the blockwise inversion formula to write the inverse-variance matrix as:

$$\begin{equation} \begin{aligned} \boldsymbol{\Sigma}^{-1} = \begin{bmatrix} \boldsymbol{\Sigma}_{11} & \boldsymbol{\Sigma}_{12} \\ \boldsymbol{\Sigma}_{21} & \boldsymbol{\Sigma}_{22} \\ \end{bmatrix}^{-1} = \begin{bmatrix} \boldsymbol{\Sigma}_{11}^* & \boldsymbol{\Sigma}_{12}^* \\ \boldsymbol{\Sigma}_{21}^* & \boldsymbol{\Sigma}_{22}^* \\ \end{bmatrix}, \end{aligned} \end{equation}$$

where:

$$\begin{equation} \begin{aligned} \begin{matrix} \boldsymbol{\Sigma}_{11}^* = \boldsymbol{\Sigma}_*^{-1} \text{ } \quad \quad \quad \quad & & & & & \boldsymbol{\Sigma}_{12}^* = -\boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1}, \quad \quad \quad \\[6pt] \boldsymbol{\Sigma}_{21}^* = - \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} & & & & & \boldsymbol{\Sigma}_{22}^* = \boldsymbol{\Sigma}_{22}^{-1} + \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1}. \text{ } \\[6pt] \end{matrix} \end{aligned} \end{equation}$$

Using this formula we can now write the Mahalanobis distance as:

$$\begin{equation} \begin{aligned} (\boldsymbol{y} &- \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) \\[6pt] &= \begin{bmatrix} \boldsymbol{y}_1 - \boldsymbol{\mu}_1 \\ \boldsymbol{y}_2 - \boldsymbol{\mu}_2 \end{bmatrix}^\text{T} \begin{bmatrix} \boldsymbol{\Sigma}_{11}^* & \boldsymbol{\Sigma}_{12}^* \\ \boldsymbol{\Sigma}_{21}^* & \boldsymbol{\Sigma}_{22}^* \\ \end{bmatrix} \begin{bmatrix} \boldsymbol{y}_1 - \boldsymbol{\mu}_1 \\ \boldsymbol{y}_2 - \boldsymbol{\mu}_2 \end{bmatrix} \\[6pt] &= \quad (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_{11}^* (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_{12}^* (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{21}^* (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^* (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= \quad (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) - (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad - (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= (\boldsymbol{y}_1 - (\boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)))^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - (\boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2))) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) , \\[6pt] \end{aligned} \end{equation}$$

where $\boldsymbol{\mu}_* \equiv \boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)$ is the conditional mean vector. Note that this result is a general result that does not assume normality of the random vectors involved in the decomposition. It gives a useful way of decomposing the Mahalanobis distance so that it consists of a sum of quadratic forms on the marginal and conditional parts. In the conditional part the conditioning vector $\boldsymbol{y}_2$ is absorbed into the mean vector and variance matrix. To clarify the form, we repeat the equation with labelling of terms:

$$(\boldsymbol{y} - \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) = \underbrace{(\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)}_\text{Conditional Part} + \underbrace{(\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)}_\text{Marginal Part}.$$

Deriving the conditional distribution: Now that we have the above form for the Mahalanobis distance, the rest is easy. We have:

$$\begin{equation} \begin{aligned} p(\boldsymbol{y}_1 | \boldsymbol{y}_2, \boldsymbol{\mu}, \boldsymbol{\Sigma}) &\overset{\boldsymbol{y}_1}{\propto} p(\boldsymbol{y}_1 , \boldsymbol{y}_2 | \boldsymbol{\mu}, \boldsymbol{\Sigma}) \\[12pt] &= \text{N}(\boldsymbol{y} | \boldsymbol{\mu}, \boldsymbol{\Sigma}) \\[10pt] &\overset{\boldsymbol{y}_1}{\propto} \exp \Big( - \frac{1}{2} (\boldsymbol{y} - \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) \Big) \\[6pt] &\overset{\boldsymbol{y}_1}{\propto} \exp \Big( - \frac{1}{2} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*) \Big) \\[6pt] &\overset{\boldsymbol{y}_1}{\propto}\text{N}(\boldsymbol{y}_1 | \boldsymbol{\mu}_*, \boldsymbol{\Sigma}_*). \\[6pt] \end{aligned} \end{equation}$$

This establishes that the conditional distribution is also multivariate normal, with the specified conditional mean vector and conditional variance matrix.

Changed several Sigma_12 Sigma^* into Sigma_21 Sigma^*
Source Link
edit approved Nov 23, 2021 at 16:29
Vidas
edit approved Nov 23, 2021 at 16:29
Vidas
  • 3
  • 1

The answer by Macro is great, but here is an even simpler way that does not require you to use any outside theorem asserting the conditional distribution. It involves writing the Mahanalobis distance in a form that separates the argument variable for the conditioning statement, and then factorising the normal density accordingly.

Rewriting the Mahanalobis distance for a conditional vector: This derivation uses a matrix inversion formula that uses the Schur complement $\boldsymbol{\Sigma}_* \equiv \boldsymbol{\Sigma}_{11} - \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21}$. We first use the blockwise inversion formula to write the inverse-variance matrix as:

$$\begin{equation} \begin{aligned} \boldsymbol{\Sigma}^{-1} = \begin{bmatrix} \boldsymbol{\Sigma}_{11} & \boldsymbol{\Sigma}_{12} \\ \boldsymbol{\Sigma}_{21} & \boldsymbol{\Sigma}_{22} \\ \end{bmatrix}^{-1} = \begin{bmatrix} \boldsymbol{\Sigma}_{11}^* & \boldsymbol{\Sigma}_{12}^* \\ \boldsymbol{\Sigma}_{21}^* & \boldsymbol{\Sigma}_{22}^* \\ \end{bmatrix}, \end{aligned} \end{equation}$$

where:

$$\begin{equation} \begin{aligned} \begin{matrix} \boldsymbol{\Sigma}_{11}^* = \boldsymbol{\Sigma}_*^{-1} \text{ } \quad \quad \quad \quad & & & & & \boldsymbol{\Sigma}_{12}^* = -\boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1}, \quad \quad \quad \\[6pt] \boldsymbol{\Sigma}_{21}^* = - \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_*^{-1} & & & & & \boldsymbol{\Sigma}_{22}^* = \boldsymbol{\Sigma}_{22}^{-1} + \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1}. \text{ } \\[6pt] \end{matrix} \end{aligned} \end{equation}$$$$\begin{equation} \begin{aligned} \begin{matrix} \boldsymbol{\Sigma}_{11}^* = \boldsymbol{\Sigma}_*^{-1} \text{ } \quad \quad \quad \quad & & & & & \boldsymbol{\Sigma}_{12}^* = -\boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1}, \quad \quad \quad \\[6pt] \boldsymbol{\Sigma}_{21}^* = - \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} & & & & & \boldsymbol{\Sigma}_{22}^* = \boldsymbol{\Sigma}_{22}^{-1} + \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1}. \text{ } \\[6pt] \end{matrix} \end{aligned} \end{equation}$$

Using this formula we can now write the Mahanalobis distance as:

$$\begin{equation} \begin{aligned} (\boldsymbol{y} &- \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) \\[6pt] &= \begin{bmatrix} \boldsymbol{y}_1 - \boldsymbol{\mu}_1 \\ \boldsymbol{y}_2 - \boldsymbol{\mu}_2 \end{bmatrix}^\text{T} \begin{bmatrix} \boldsymbol{\Sigma}_{11}^* & \boldsymbol{\Sigma}_{12}^* \\ \boldsymbol{\Sigma}_{21}^* & \boldsymbol{\Sigma}_{22}^* \\ \end{bmatrix} \begin{bmatrix} \boldsymbol{y}_1 - \boldsymbol{\mu}_1 \\ \boldsymbol{y}_2 - \boldsymbol{\mu}_2 \end{bmatrix} \\[6pt] &= \quad (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_{11}^* (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_{12}^* (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{21}^* (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^* (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= \quad (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) - (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad - (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= (\boldsymbol{y}_1 - (\boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)))^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - (\boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2))) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) , \\[6pt] \end{aligned} \end{equation}$$$$\begin{equation} \begin{aligned} (\boldsymbol{y} &- \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) \\[6pt] &= \begin{bmatrix} \boldsymbol{y}_1 - \boldsymbol{\mu}_1 \\ \boldsymbol{y}_2 - \boldsymbol{\mu}_2 \end{bmatrix}^\text{T} \begin{bmatrix} \boldsymbol{\Sigma}_{11}^* & \boldsymbol{\Sigma}_{12}^* \\ \boldsymbol{\Sigma}_{21}^* & \boldsymbol{\Sigma}_{22}^* \\ \end{bmatrix} \begin{bmatrix} \boldsymbol{y}_1 - \boldsymbol{\mu}_1 \\ \boldsymbol{y}_2 - \boldsymbol{\mu}_2 \end{bmatrix} \\[6pt] &= \quad (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_{11}^* (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_{12}^* (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{21}^* (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^* (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= \quad (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) - (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad - (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= (\boldsymbol{y}_1 - (\boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)))^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - (\boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2))) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) , \\[6pt] \end{aligned} \end{equation}$$

where $\boldsymbol{\mu}_* \equiv \boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)$ is the conditional mean vector. Note that this result is a general result that does not assume normality of the random vectors involved in the decomposition. It gives a useful way of decomposing the Mahanalobis distance so that it consists of a sum of quadratic forms on the marginal and conditional parts. In the conditional part the conditioning vector $\boldsymbol{y}_2$ is absorbed into the mean vector and variance matrix. To clarify the form, we repeat the equation with labelling of terms:

$$(\boldsymbol{y} - \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) = \underbrace{(\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)}_\text{Conditional Part} + \underbrace{(\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)}_\text{Marginal Part}.$$

Deriving the conditional distribution: Now that we have the above form for the Mahanalobis distance, the rest is easy. We have:

$$\begin{equation} \begin{aligned} p(\boldsymbol{y}_1 | \boldsymbol{y}_2, \boldsymbol{\mu}, \boldsymbol{\Sigma}) &\overset{\boldsymbol{y}_1}{\propto} p(\boldsymbol{y}_1 , \boldsymbol{y}_2 | \boldsymbol{\mu}, \boldsymbol{\Sigma}) \\[12pt] &= \text{N}(\boldsymbol{y} | \boldsymbol{\mu}, \boldsymbol{\Sigma}) \\[10pt] &\overset{\boldsymbol{y}_1}{\propto} \exp \Big( - \frac{1}{2} (\boldsymbol{y} - \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) \Big) \\[6pt] &\overset{\boldsymbol{y}_1}{\propto} \exp \Big( - \frac{1}{2} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*) \Big) \\[6pt] &\overset{\boldsymbol{y}_1}{\propto}\text{N}(\boldsymbol{y}_1 | \boldsymbol{\mu}_*, \boldsymbol{\Sigma}_*). \\[6pt] \end{aligned} \end{equation}$$

This establishes that the conditional distribution is also multivariate normal, with the specified conditional mean vector and conditional variance matrix.

The answer by Macro is great, but here is an even simpler way that does not require you to use any outside theorem asserting the conditional distribution. It involves writing the Mahanalobis distance in a form that separates the argument variable for the conditioning statement, and then factorising the normal density accordingly.

Rewriting the Mahanalobis distance for a conditional vector: This derivation uses a matrix inversion formula that uses the Schur complement $\boldsymbol{\Sigma}_* \equiv \boldsymbol{\Sigma}_{11} - \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21}$. We first use the blockwise inversion formula to write the inverse-variance matrix as:

$$\begin{equation} \begin{aligned} \boldsymbol{\Sigma}^{-1} = \begin{bmatrix} \boldsymbol{\Sigma}_{11} & \boldsymbol{\Sigma}_{12} \\ \boldsymbol{\Sigma}_{21} & \boldsymbol{\Sigma}_{22} \\ \end{bmatrix}^{-1} = \begin{bmatrix} \boldsymbol{\Sigma}_{11}^* & \boldsymbol{\Sigma}_{12}^* \\ \boldsymbol{\Sigma}_{21}^* & \boldsymbol{\Sigma}_{22}^* \\ \end{bmatrix}, \end{aligned} \end{equation}$$

where:

$$\begin{equation} \begin{aligned} \begin{matrix} \boldsymbol{\Sigma}_{11}^* = \boldsymbol{\Sigma}_*^{-1} \text{ } \quad \quad \quad \quad & & & & & \boldsymbol{\Sigma}_{12}^* = -\boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1}, \quad \quad \quad \\[6pt] \boldsymbol{\Sigma}_{21}^* = - \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_*^{-1} & & & & & \boldsymbol{\Sigma}_{22}^* = \boldsymbol{\Sigma}_{22}^{-1} + \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1}. \text{ } \\[6pt] \end{matrix} \end{aligned} \end{equation}$$

Using this formula we can now write the Mahanalobis distance as:

$$\begin{equation} \begin{aligned} (\boldsymbol{y} &- \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) \\[6pt] &= \begin{bmatrix} \boldsymbol{y}_1 - \boldsymbol{\mu}_1 \\ \boldsymbol{y}_2 - \boldsymbol{\mu}_2 \end{bmatrix}^\text{T} \begin{bmatrix} \boldsymbol{\Sigma}_{11}^* & \boldsymbol{\Sigma}_{12}^* \\ \boldsymbol{\Sigma}_{21}^* & \boldsymbol{\Sigma}_{22}^* \\ \end{bmatrix} \begin{bmatrix} \boldsymbol{y}_1 - \boldsymbol{\mu}_1 \\ \boldsymbol{y}_2 - \boldsymbol{\mu}_2 \end{bmatrix} \\[6pt] &= \quad (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_{11}^* (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_{12}^* (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{21}^* (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^* (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= \quad (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) - (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad - (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= (\boldsymbol{y}_1 - (\boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)))^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - (\boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2))) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) , \\[6pt] \end{aligned} \end{equation}$$

where $\boldsymbol{\mu}_* \equiv \boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)$ is the conditional mean vector. Note that this result is a general result that does not assume normality of the random vectors involved in the decomposition. It gives a useful way of decomposing the Mahanalobis distance so that it consists of a sum of quadratic forms on the marginal and conditional parts. In the conditional part the conditioning vector $\boldsymbol{y}_2$ is absorbed into the mean vector and variance matrix. To clarify the form, we repeat the equation with labelling of terms:

$$(\boldsymbol{y} - \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) = \underbrace{(\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)}_\text{Conditional Part} + \underbrace{(\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)}_\text{Marginal Part}.$$

Deriving the conditional distribution: Now that we have the above form for the Mahanalobis distance, the rest is easy. We have:

$$\begin{equation} \begin{aligned} p(\boldsymbol{y}_1 | \boldsymbol{y}_2, \boldsymbol{\mu}, \boldsymbol{\Sigma}) &\overset{\boldsymbol{y}_1}{\propto} p(\boldsymbol{y}_1 , \boldsymbol{y}_2 | \boldsymbol{\mu}, \boldsymbol{\Sigma}) \\[12pt] &= \text{N}(\boldsymbol{y} | \boldsymbol{\mu}, \boldsymbol{\Sigma}) \\[10pt] &\overset{\boldsymbol{y}_1}{\propto} \exp \Big( - \frac{1}{2} (\boldsymbol{y} - \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) \Big) \\[6pt] &\overset{\boldsymbol{y}_1}{\propto} \exp \Big( - \frac{1}{2} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*) \Big) \\[6pt] &\overset{\boldsymbol{y}_1}{\propto}\text{N}(\boldsymbol{y}_1 | \boldsymbol{\mu}_*, \boldsymbol{\Sigma}_*). \\[6pt] \end{aligned} \end{equation}$$

This establishes that the conditional distribution is also multivariate normal, with the specified conditional mean vector and conditional variance matrix.

The answer by Macro is great, but here is an even simpler way that does not require you to use any outside theorem asserting the conditional distribution. It involves writing the Mahanalobis distance in a form that separates the argument variable for the conditioning statement, and then factorising the normal density accordingly.

Rewriting the Mahanalobis distance for a conditional vector: This derivation uses a matrix inversion formula that uses the Schur complement $\boldsymbol{\Sigma}_* \equiv \boldsymbol{\Sigma}_{11} - \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21}$. We first use the blockwise inversion formula to write the inverse-variance matrix as:

$$\begin{equation} \begin{aligned} \boldsymbol{\Sigma}^{-1} = \begin{bmatrix} \boldsymbol{\Sigma}_{11} & \boldsymbol{\Sigma}_{12} \\ \boldsymbol{\Sigma}_{21} & \boldsymbol{\Sigma}_{22} \\ \end{bmatrix}^{-1} = \begin{bmatrix} \boldsymbol{\Sigma}_{11}^* & \boldsymbol{\Sigma}_{12}^* \\ \boldsymbol{\Sigma}_{21}^* & \boldsymbol{\Sigma}_{22}^* \\ \end{bmatrix}, \end{aligned} \end{equation}$$

where:

$$\begin{equation} \begin{aligned} \begin{matrix} \boldsymbol{\Sigma}_{11}^* = \boldsymbol{\Sigma}_*^{-1} \text{ } \quad \quad \quad \quad & & & & & \boldsymbol{\Sigma}_{12}^* = -\boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1}, \quad \quad \quad \\[6pt] \boldsymbol{\Sigma}_{21}^* = - \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} & & & & & \boldsymbol{\Sigma}_{22}^* = \boldsymbol{\Sigma}_{22}^{-1} + \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1}. \text{ } \\[6pt] \end{matrix} \end{aligned} \end{equation}$$

Using this formula we can now write the Mahanalobis distance as:

$$\begin{equation} \begin{aligned} (\boldsymbol{y} &- \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) \\[6pt] &= \begin{bmatrix} \boldsymbol{y}_1 - \boldsymbol{\mu}_1 \\ \boldsymbol{y}_2 - \boldsymbol{\mu}_2 \end{bmatrix}^\text{T} \begin{bmatrix} \boldsymbol{\Sigma}_{11}^* & \boldsymbol{\Sigma}_{12}^* \\ \boldsymbol{\Sigma}_{21}^* & \boldsymbol{\Sigma}_{22}^* \\ \end{bmatrix} \begin{bmatrix} \boldsymbol{y}_1 - \boldsymbol{\mu}_1 \\ \boldsymbol{y}_2 - \boldsymbol{\mu}_2 \end{bmatrix} \\[6pt] &= \quad (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_{11}^* (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_{12}^* (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{21}^* (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^* (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= \quad (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) - (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad - (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= (\boldsymbol{y}_1 - (\boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)))^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - (\boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2))) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) , \\[6pt] \end{aligned} \end{equation}$$

where $\boldsymbol{\mu}_* \equiv \boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)$ is the conditional mean vector. Note that this result is a general result that does not assume normality of the random vectors involved in the decomposition. It gives a useful way of decomposing the Mahanalobis distance so that it consists of a sum of quadratic forms on the marginal and conditional parts. In the conditional part the conditioning vector $\boldsymbol{y}_2$ is absorbed into the mean vector and variance matrix. To clarify the form, we repeat the equation with labelling of terms:

$$(\boldsymbol{y} - \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) = \underbrace{(\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)}_\text{Conditional Part} + \underbrace{(\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)}_\text{Marginal Part}.$$

Deriving the conditional distribution: Now that we have the above form for the Mahanalobis distance, the rest is easy. We have:

$$\begin{equation} \begin{aligned} p(\boldsymbol{y}_1 | \boldsymbol{y}_2, \boldsymbol{\mu}, \boldsymbol{\Sigma}) &\overset{\boldsymbol{y}_1}{\propto} p(\boldsymbol{y}_1 , \boldsymbol{y}_2 | \boldsymbol{\mu}, \boldsymbol{\Sigma}) \\[12pt] &= \text{N}(\boldsymbol{y} | \boldsymbol{\mu}, \boldsymbol{\Sigma}) \\[10pt] &\overset{\boldsymbol{y}_1}{\propto} \exp \Big( - \frac{1}{2} (\boldsymbol{y} - \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) \Big) \\[6pt] &\overset{\boldsymbol{y}_1}{\propto} \exp \Big( - \frac{1}{2} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*) \Big) \\[6pt] &\overset{\boldsymbol{y}_1}{\propto}\text{N}(\boldsymbol{y}_1 | \boldsymbol{\mu}_*, \boldsymbol{\Sigma}_*). \\[6pt] \end{aligned} \end{equation}$$

This establishes that the conditional distribution is also multivariate normal, with the specified conditional mean vector and conditional variance matrix.

Edited to remove notational inconsistency
Source Link
Ben
Ben
  • 132.9k
  • 7
  • 255
  • 588
added 33 characters in body
Source Link
Ben
Ben
  • 132.9k
  • 7
  • 255
  • 588
Edit to correct error in decomposition
Source Link
Ben
Ben
  • 132.9k
  • 7
  • 255
  • 588
added 52 characters in body
Source Link
Ben
Ben
  • 132.9k
  • 7
  • 255
  • 588
Source Link
Ben
Ben
  • 132.9k
  • 7
  • 255
  • 588