What is the conditional probability of variables in a multivariate gaussian?

Question

Given $Y = [Y_1, Y_2, \ldots, Y_n]^T \sim N(0, \Sigma)$. That is $f_{Y}(y_1, y_2, \ldots, y_n)$ is a multivariate gaussian with mean $0$ and covariance matrix $\Sigma$.

I'm asked to compute the conditional correlation of $y_i, y_j$ given $y_k$, that is $\rho(y_i, y_j | y_k)$. I know that in order to do this, I need to compute $$E[y_i \cdot y_j|y_k] = \int_{-\infty}^{\infty} y_i \,y_k \,\, f(y_i, y_j | y_k) \,d$$.

How can I compute $f(y_i, y_j | y_k)$? I know that this expression can be computed by $$f(y_i, y_j | y_k) = \frac{f(y_i, y_j)}{f(y_k)}$$

But I'm not sure how to find these expressions in closed form.

Answer 1

The Schur Complement, is the goto for this kind of computation. Let $Y \sim \mathcal{N}(0,\Sigma)$, where $Y$ and $\Sigma$ are organized such that the last element and row/column belong to the variable you wish to condition on. Then, $Y_{-k}|Y_k$ is distributed, $\mathcal{N}\left(\mu_{-k}, \Sigma_{-k}\right)$, where

• $Y_{-k}$ consists of the elements of $Y$ not including $k$;

• $\Sigma = \left( \begin{array}{rr} A & B\\ B^{T} & C\\ \end{array} \right)$ in slightly R'ish parlance $A = \Sigma[-k,-k]$, $B=\Sigma[-k,k]$, and $C = \Sigma[k,k]$;

• $\Sigma_{-k} = A - BC^{-1}B^T$;

• and $\mu_{-k} = BC^{-1}Y$.

The covariance of $Y_i$ and $Y_j$ given $Y_k$ can then be retrieved from $\Sigma_{-k}$. A fast way to calculate this numerically is through the sweep function (Dempster 1969).

Dempster, A.P. (1969). Elements of continuous multivariate analysis. Reading, MA: Addison- Wesley.