We have a multivariate normal vector ${\boldsymbol Y} \sim \mathcal{N}(\boldsymbol\mu, \Sigma)$. Consider partitioning $\boldsymbol\mu$ and ${\boldsymbol Y}$ into $$\boldsymbol\mu = \begin{bmatrix} \boldsymbol\mu_1 \\ \boldsymbol\mu_2 \end{bmatrix} $$ $${\boldsymbol Y}=\begin{bmatrix}{\boldsymbol y}_1 \\ {\boldsymbol y}_2 \end{bmatrix}$$

with a similar partition of $\Sigma$ into $$ \begin{bmatrix} \Sigma_{11} & \Sigma_{12}\\ \Sigma_{21} & \Sigma_{22} \end{bmatrix} $$ Then, $({\boldsymbol y}_1|{\boldsymbol y}_2={\boldsymbol a})$, the conditional distribution of the first partition given the second, is $\mathcal{N}(\overline{\boldsymbol\mu},\overline{\Sigma})$, with mean
$$ \overline{\boldsymbol\mu}=\boldsymbol\mu_1+\Sigma_{12}{\Sigma_{22}}^{-1}({\boldsymbol a}-\boldsymbol\mu_2) $$ and covariance matrix $$ \overline{\Sigma}=\Sigma_{11}-\Sigma_{12}{\Sigma_{22}}^{-1}\Sigma_{21}$$

Actually these results are provided in Wikipedia too, but I have no idea how the $\overline{\boldsymbol\mu}$ and $\overline{\Sigma}$ is derived. These results are crucial, since they are important statistical formula for deriving Kalman filters. Would anyone provide me a derivation steps of deriving $\overline{\boldsymbol\mu}$ and $\overline{\Sigma}$ ?

  • 44
    $\begingroup$ The idea is to use the definition of conditional density $f(y_1\vert y_2=a)=\dfrac{f_{Y_1,Y_2}(y_1,a)}{f_{Y_2}(a)}$. You know that the joint $f_{Y_1,Y_2}$ is a bivariate normal and that the marginal $f_{Y_2}$ is a normal then you just have to replace the values and do the unpleasant algebra. These notes might be of some help. Here is the full proof. $\endgroup$
    – user10525
    Commented Jun 16, 2012 at 18:16
  • 1
    $\begingroup$ Your second link answers the question (+1). Why not put it as an answer @Procrastinator? $\endgroup$
    – gui11aume
    Commented Jun 16, 2012 at 22:54
  • 1
    $\begingroup$ I hadn't realized it, but I think I was implicitly using this equation in a conditional PCA. The conditional PCA requires a transformation $\left(I-A'\left(AA'\right)^{-1}A\right)\Sigma$ that is effectively calculating the conditional covariance matrix given some choice of A. $\endgroup$
    – John
    Commented Jul 2, 2012 at 15:49
  • $\begingroup$ @Procrastinator - your approach actually requires the knowledge of the Woodbury matrix identity, and the knowledge of block-wise matrix inversion. These result in unnecessarily complicated matrix algebra. $\endgroup$ Commented Jul 2, 2012 at 16:17
  • 2
    $\begingroup$ @probabilityislogic Actually the result is proved in the link I provided. But it is respectable if you find it more complicated than other methods. In addition, I was not attempting to provide an optimal solution in my comment. Also, my comment was previous to Macro's answer (which I upvoted as you can see). $\endgroup$
    – user10525
    Commented Jul 2, 2012 at 16:25

3 Answers 3


You can prove it by explicitly calculating the conditional density by brute force, as in Procrastinator's link (+1) in the comments. But, there's also a theorem that says all conditional distributions of a multivariate normal distribution are normal. Therefore, all that's left is to calculate the mean vector and covariance matrix. I remember we derived this in a time series class in college by cleverly defining a third variable and using its properties to derive the result more simply than the brute force solution in the link (as long as you're comfortable with matrix algebra). I'm going from memory but it was something like this:

It is worth pointing out that the proof below only assumes that $\Sigma_{22}$ is nonsingular, $\Sigma_{11}$ and $\Sigma$ may well be singular.

Let ${\bf x}_{1}$ be the first partition and ${\bf x}_2$ the second. Now define ${\bf z} = {\bf x}_1 + {\bf A} {\bf x}_2 $ where ${\bf A} = -\Sigma_{12} \Sigma^{-1}_{22}$. Now we can write

\begin{align*} {\rm cov}({\bf z}, {\bf x}_2) &= {\rm cov}( {\bf x}_{1}, {\bf x}_2 ) + {\rm cov}({\bf A}{\bf x}_2, {\bf x}_2) \\ &= \Sigma_{12} + {\bf A} {\rm var}({\bf x}_2) \\ &= \Sigma_{12} - \Sigma_{12} \Sigma^{-1}_{22} \Sigma_{22} \\ &= 0 \end{align*}

Therefore ${\bf z}$ and ${\bf x}_2$ are uncorrelated and, since they are jointly normal, they are independent. Now, clearly $E({\bf z}) = {\boldsymbol \mu}_1 + {\bf A} {\boldsymbol \mu}_2$, therefore it follows that

\begin{align*} E({\bf x}_1 | {\bf x}_2) &= E( {\bf z} - {\bf A} {\bf x}_2 | {\bf x}_2) \\ & = E({\bf z}|{\bf x}_2) - E({\bf A}{\bf x}_2|{\bf x}_2) \\ & = E({\bf z}) - {\bf A}{\bf x}_2 \\ & = {\boldsymbol \mu}_1 + {\bf A} ({\boldsymbol \mu}_2 - {\bf x}_2) \\ & = {\boldsymbol \mu}_1 + \Sigma_{12} \Sigma^{-1}_{22} ({\bf x}_2- {\boldsymbol \mu}_2) \end{align*}

which proves the first part. For the covariance matrix, note that

\begin{align*} {\rm var}({\bf x}_1|{\bf x}_2) &= {\rm var}({\bf z} - {\bf A} {\bf x}_2 | {\bf x}_2) \\ &= {\rm var}({\bf z}|{\bf x}_2) + {\rm var}({\bf A} {\bf x}_2 | {\bf x}_2) - {\bf A}{\rm cov}({\bf z}, -{\bf x}_2) - {\rm cov}({\bf z}, -{\bf x}_2) {\bf A}' \\ &= {\rm var}({\bf z}|{\bf x}_2) \\ &= {\rm var}({\bf z}) \end{align*}

Now we're almost done:

\begin{align*} {\rm var}({\bf x}_1|{\bf x}_2) = {\rm var}( {\bf z} ) &= {\rm var}( {\bf x}_1 + {\bf A} {\bf x}_2 ) \\ &= {\rm var}( {\bf x}_1 ) + {\bf A} {\rm var}( {\bf x}_2 ) {\bf A}' + {\bf A} {\rm cov}({\bf x}_1,{\bf x}_2) + {\rm cov}({\bf x}_2,{\bf x}_1) {\bf A}' \\ &= \Sigma_{11} +\Sigma_{12} \Sigma^{-1}_{22} \Sigma_{22}\Sigma^{-1}_{22}\Sigma_{21} - 2 \Sigma_{12} \Sigma_{22}^{-1} \Sigma_{21} \\ &= \Sigma_{11} +\Sigma_{12} \Sigma^{-1}_{22}\Sigma_{21} - 2 \Sigma_{12} \Sigma_{22}^{-1} \Sigma_{21} \\ &= \Sigma_{11} -\Sigma_{12} \Sigma^{-1}_{22}\Sigma_{21} \end{align*}

which proves the second part.

Note: For those not very familiar with the matrix algebra used here, this is an excellent resource.

Edit: One property used here this is not in the matrix cookbook (good catch @FlyingPig) is property 6 on the wikipedia page about covariance matrices: which is that for two random vectors $\bf x, y$, $${\rm var}({\bf x}+{\bf y}) = {\rm var}({\bf x})+{\rm var}({\bf y}) + {\rm cov}({\bf x},{\bf y}) + {\rm cov}({\bf y},{\bf x})$$ For scalars, of course, ${\rm cov}(X,Y)={\rm cov}(Y,X)$ but for vectors they are different insofar as the matrices are arranged differently.

  • 2
    $\begingroup$ Thanks for this brilliant method! There is one matrix algebra does not seem familiar to me, where can I find the formula for opening $var(x_1+Ax_2)$? I haven't found it on the link you sent. $\endgroup$
    – Flying pig
    Commented Jun 17, 2012 at 6:35
  • 26
    $\begingroup$ This is a very good answer (+1), but could be improved in terms of the ordering of the approach. We start with saying we want a linear combination $z=Cx=C_1x_1+C_2x_2$ of the whole vector that is independent/uncorrelated with $x_2$. This is because we can use the fact that $p(z|x_2)=p(z)$ which means $var(z|x_2)=var(z)$ and $E(z|x_2)=E(z)$. These in turn lead to expressions for $var(C_1x_1|x_2)$ and $E(C_1x_1|x_2)$. This means we should take $C_1=I$. Now we require $cov(z,x_2)=\Sigma_{12}+C_2\Sigma_{22}=0$. If $\Sigma_{22}$ is invertible we then have $C_2=-\Sigma_{12}\Sigma_{22}^{-1}$. $\endgroup$ Commented Jul 2, 2012 at 16:00
  • 3
    $\begingroup$ @probabilityislogic, I'd actually never thought about the process that resulted in choosing this linear combination but your comment makes it clear that it arises naturally, considering the constraints we want to satisfy. +1! $\endgroup$
    – Macro
    Commented Jul 2, 2012 at 20:06
  • 1
    $\begingroup$ @jakeoung - it is not proving that $C_1=I$, it is setting it to this value, so that we get an expression that contains the variables we want to know about. $\endgroup$ Commented Jan 14, 2018 at 14:40
  • 1
    $\begingroup$ @jakeoung I also don't quite understand that statement. I understand in this way: If $cov(z, x_2)=0$, then $cov(C_1^{-1} z, x_2) = C_1^{-1} cov( z, x_2)=0$. So the value of $C_1$ is somehow an arbitrary scale. So we set $C_1=I$ for simplicity. $\endgroup$
    – Ken T
    Commented May 5, 2018 at 16:03

The answer by Macro is great, but here is an even simpler way that does not require you to use any outside theorem asserting the conditional distribution. It involves writing the Mahalanobis distance in a form that separates the argument variable for the conditioning statement, and then factorising the normal density accordingly.

Rewriting the Mahalanobis distance for a conditional vector: This derivation uses a matrix inversion formula that uses the Schur complement $\boldsymbol{\Sigma}_* \equiv \boldsymbol{\Sigma}_{11} - \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21}$. We first use the blockwise inversion formula to write the inverse-variance matrix as:

$$\begin{equation} \begin{aligned} \boldsymbol{\Sigma}^{-1} = \begin{bmatrix} \boldsymbol{\Sigma}_{11} & \boldsymbol{\Sigma}_{12} \\ \boldsymbol{\Sigma}_{21} & \boldsymbol{\Sigma}_{22} \\ \end{bmatrix}^{-1} = \begin{bmatrix} \boldsymbol{\Sigma}_{11}^* & \boldsymbol{\Sigma}_{12}^* \\ \boldsymbol{\Sigma}_{21}^* & \boldsymbol{\Sigma}_{22}^* \\ \end{bmatrix}, \end{aligned} \end{equation}$$


$$\begin{equation} \begin{aligned} \begin{matrix} \boldsymbol{\Sigma}_{11}^* = \boldsymbol{\Sigma}_*^{-1} \text{ } \quad \quad \quad \quad & & & & & \boldsymbol{\Sigma}_{12}^* = -\boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1}, \quad \quad \quad \\[6pt] \boldsymbol{\Sigma}_{21}^* = - \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} & & & & & \boldsymbol{\Sigma}_{22}^* = \boldsymbol{\Sigma}_{22}^{-1} + \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1}. \text{ } \\[6pt] \end{matrix} \end{aligned} \end{equation}$$

Using this formula we can now write the Mahalanobis distance as:

$$\begin{equation} \begin{aligned} (\boldsymbol{y} &- \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) \\[6pt] &= \begin{bmatrix} \boldsymbol{y}_1 - \boldsymbol{\mu}_1 \\ \boldsymbol{y}_2 - \boldsymbol{\mu}_2 \end{bmatrix}^\text{T} \begin{bmatrix} \boldsymbol{\Sigma}_{11}^* & \boldsymbol{\Sigma}_{12}^* \\ \boldsymbol{\Sigma}_{21}^* & \boldsymbol{\Sigma}_{22}^* \\ \end{bmatrix} \begin{bmatrix} \boldsymbol{y}_1 - \boldsymbol{\mu}_1 \\ \boldsymbol{y}_2 - \boldsymbol{\mu}_2 \end{bmatrix} \\[6pt] &= \quad (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_{11}^* (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_{12}^* (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{21}^* (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^* (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= \quad (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) - (\boldsymbol{y}_1 - \boldsymbol{\mu}_1)^\text{T} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad - (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_1) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} \boldsymbol{\Sigma}_{21} \boldsymbol{\Sigma}_*^{-1} \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= (\boldsymbol{y}_1 - (\boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)))^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - (\boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2))) \\[6pt] &\quad + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) \\[6pt] &= (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*) + (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2) , \\[6pt] \end{aligned} \end{equation}$$

where $\boldsymbol{\mu}_* \equiv \boldsymbol{\mu}_1 + \boldsymbol{\Sigma}_{12} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)$ (which will be the conditional mean vector in the normal case). Note that this result is a general result that does not assume normality of the random vectors involved in the decomposition. It gives a useful way of decomposing the Mahalanobis distance so that it consists of a sum of quadratic forms on the marginal and conditional parts. In the conditional part the conditioning vector $\boldsymbol{y}_2$ is absorbed into the mean vector and variance matrix. To clarify the form, we repeat the equation with labelling of terms:

$$(\boldsymbol{y} - \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) = \underbrace{(\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)}_\text{Conditional Part} + \underbrace{(\boldsymbol{y}_2 - \boldsymbol{\mu}_2)^\text{T} \boldsymbol{\Sigma}_{22}^{-1} (\boldsymbol{y}_2 - \boldsymbol{\mu}_2)}_\text{Marginal Part}.$$

Deriving the conditional distribution: Now that we have the above form for the Mahalanobis distance, the rest is easy. We have:

$$\begin{equation} \begin{aligned} p(\boldsymbol{y}_1 | \boldsymbol{y}_2, \boldsymbol{\mu}, \boldsymbol{\Sigma}) &\overset{\boldsymbol{y}_1}{\propto} p(\boldsymbol{y}_1 , \boldsymbol{y}_2 | \boldsymbol{\mu}, \boldsymbol{\Sigma}) \\[12pt] &= \text{N}(\boldsymbol{y} | \boldsymbol{\mu}, \boldsymbol{\Sigma}) \\[10pt] &\overset{\boldsymbol{y}_1}{\propto} \exp \Big( - \frac{1}{2} (\boldsymbol{y} - \boldsymbol{\mu})^\text{T} \boldsymbol{\Sigma}^{-1} (\boldsymbol{y} - \boldsymbol{\mu}) \Big) \\[6pt] &\overset{\boldsymbol{y}_1}{\propto} \exp \Big( - \frac{1}{2} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*)^\text{T} \boldsymbol{\Sigma}_*^{-1} (\boldsymbol{y}_1 - \boldsymbol{\mu}_*) \Big) \\[6pt] &\overset{\boldsymbol{y}_1}{\propto}\text{N}(\boldsymbol{y}_1 | \boldsymbol{\mu}_*, \boldsymbol{\Sigma}_*). \\[6pt] \end{aligned} \end{equation}$$

This establishes that the conditional distribution is also multivariate normal, with the specified conditional mean vector and conditional variance matrix.

  • $\begingroup$ Hi Ben. I am sorry for another question. Will the above marginal distribution hold if we don't assume normal distribution for $y_1$ and $y_2$. Then what's the conditional distribution for $y_1$ conditional $y_2$ without normal distribution? Or, is it possible to calculate the expectation and variance of $y_1$ conditional $y_2$ without normal distribution following your setup without normality assumption. Deeply appreciate for your help! Thanks $\endgroup$ Commented Nov 24, 2021 at 16:48
  • 1
    $\begingroup$ @CharlesChou: No, the moments $\boldsymbol{\mu}_*$ and $\boldsymbol{\Sigma}_*$ will not generally hold outside the normal distribution. (Also, note that the above is a conditional distribution, not a marginal distribution.) $\endgroup$
    – Ben
    Commented Nov 24, 2021 at 20:24

Since I regularly come back to this, here is my attempt to reduce the gore of Ben's answer (wether or not you think this is true will depend on your tastes).

With this approach it turns out to be a bit easier to calculate $Y_2\mid Y_1$ so weare going to do that instead. Notationwise we define $$ \bar{y} := y-\mu $$ So that we can essentially rid ourselves of the mean without loss of generality.

Cholesky based proof

The main insight for this proof is the use of the cholesky decomposition of the covariance matrix, i.e. $\Sigma = LL^T$ where $L$ is a lower triangular matrix.

Why is this useful? Well, behold: $$ \bar{y}^T \Sigma^{-1} \bar{y} = \bar{y} L^{-T}L^{-1} \bar{y} = \|L^{-1}\bar{y}\|^2. $$

Let us calculate this inverse $$ \begin{bmatrix} \bar{y}_1\\ \bar{y}_2 \end{bmatrix} = \begin{bmatrix} L_{11} & 0\\ L_{21} & L_{22} \end{bmatrix}\begin{bmatrix} x_1\\ x_2 \end{bmatrix} $$ This implies \begin{align*} x_1 &= L_{11}^{-1}\bar{y}_1 \\ x_2 &= L_{22}^{-1}(\bar{y}_2 - L_{21}x_1) \\ &= L_{22}^{-1}\bigl( y_2 - \underbrace{(\mu_2 + L_{21}L_{11}^{-1}\bar{y}_1)}_{=:\mu_{2\mid 1}}\bigr) \end{align*}

So we have \begin{align*} \bar{y}^T \Sigma^{-1} \bar{y} &= \|x_1\|^2 &+\quad& \|x_2\|^2 \\ &= \underbrace{\bar{y}_1^T L_{11}^{-T}L_{11}^{-1}\bar{y}_1}_{\text{marginal part}} &+\quad& \underbrace{(y_2- \mu_{2\mid 1})^T L_{22}^{-T}L_{22}^{-1}(y_2- \mu_{2\mid 1})}_{\text{conditional part}} \end{align*} Now we are almost done, we simply need to get back to the covariance matrix version.

For this consider $$ \Sigma = \begin{bmatrix} \Sigma_{11} & \Sigma_{12}\\ \Sigma_{21} & \Sigma_{22} \end{bmatrix} = \begin{bmatrix} L_{11}L_{11}^T & L_{11}L_{21}^T \\ L_{21}L_{11}^T & L_{21}L_{21}^T + L_{22}L_{22}^T \end{bmatrix} = LL^T $$ with the lower triangular block matrix $$ L = \begin{bmatrix} L_{11} & 0\\ L_{21} & L_{22} \end{bmatrix}. $$ From this we obtain \begin{align*} && L_{11}L_{11}^T &= \Sigma_{11} \\ \Sigma_{2\mid 1} &:=& L_{22}L_{22}^T &= \Sigma_{22} - L_{21}L_{21}^T \\ && & = \Sigma_{22} - \underbrace{L_{21}L_{11}^T}_{=\Sigma_{21}} \underbrace{L_{11}^{-T} L_{11}^{-1}}_{=\Sigma_{11}^{-1}} \underbrace{L_{11}L_{21}^T}_{=\Sigma_{12}} \\ \mu_{2\mid 1} &:=& \mu_2 + L_{21}L_{11}^{-1}\bar{y}_1 &= \mu_2 + \underbrace{L_{21}L_{11}^T}_{=\Sigma_{21}} \underbrace{L_{11}^{-T}L_{11}^{-1}}_{=\Sigma_{11}^{-1}}\bar{y}_1 \end{align*}

Putting everything together we have \begin{align*} (y-\mu)^T \Sigma^{-1} (y-\mu) &= \underbrace{(y_1-\mu_1)^T \Sigma_{11}^{-1} (y_1-\mu_1)}_{\text{marginal part}} + \underbrace{(y_2-\mu_{2\mid 1})^T \Sigma_{2\mid 1}^{-1} (y_2-\mu_{2\mid 1})}_{\text{conditional part}} \end{align*} with \begin{align*} \Sigma_{2\mid 1} &= \Sigma_{22} - \Sigma_{21}\Sigma_{11}^{-1}\Sigma_{12} \\ \mu_{2\mid 1} &= \mu_2 + \Sigma_{21}\Sigma_{11}^{-1}(y_1-\mu_1) \end{align*} Integrating out the marginal part leaves the conditional density (where we can ignore the constants because we know at the end we will get a density).

A note on the intuition

Notice that if $Y\sim \mathcal{N}(0,\Sigma)$, then $L^{-1}Y \sim \mathcal{N}(0, \mathbb{I})$. So we essentially removed all depdence using $L^{-1}$. Or starting from iid standard normal $X\sim\mathcal{N}(0, \mathbb{I})$ we can obtain our distribution $$Y=L X \sim \mathcal{N}(0,\Sigma).$$ Due to the lower triangular nature, only $X_1$ is used for $Y_1$. And by $L_{11}$ we can transform back and forth between $X_1$ and $Y_1$. So now the question is, what do we expect of $Y_2$ conditional on $Y_1$ (or equivalently $X_1$)? Well we know that $$ Y_2 = \underbrace{L_{21} X_1}_{\text{known}} + \underbrace{L_{22} X_2}_{\text{unknown}} $$ In essence $X_1$ is the old randomness and $X_2$ is the new randomness. So here the conditional expectation and conditional variance come from. That is why the conditional expectation is given by $$ \mathbb{E}[Y_2\mid Y_1] = L_{21}X_1 = L_{21}L_{11}^{-1}Y_1 $$ And why the conditional variance is given by $$ L_{22}L_{22}^T $$ which we then had to translate back into the covariance form.


Not the answer you're looking for? Browse other questions tagged or ask your own question.