How to get conditional variance from Schur complement?

Question

Suppose you have vectors X and Y with covariance matrix $V = \left( \begin{array}{cc} A & B \\ B^T & C \end{array} \right)$. This Wikipedia article says that $Var(X | Y) = A - BC^{-1}B^T$, the Schur Complement of C in V.

Furthermore, conditional (in)dependence between X and Y can be found in the inverse of the covariance matrix $V^{-1}$. Earlier in the Wiki page they show that the inverse of a matrix is a function of the Schur Complement.

I'm trying to piece these two pieces together. First, how do we show $Var(X | Y) = A - BC^{-1}B^T$, and second: if $V^{-1} = 0$, how do we show that the Schur complement is necessarily 0, and hence $Var(X | Y) = 0$?

Answer 1

Assume WLOG that everything is mean $0$. If you know the formula for the inverse of a matrix in block form, then it should be as simple as checking that

$$(X, Y)^T V^{-1} (X, Y) = X^T A^{-1} X + (Y - \mu_{Y | X})^T S^{-1} (Y - \mu_{Y|X}) + c$$

where $S$ is the Schur complement and $c$ is a constant and $\mu_{Y|X} = B^TA^{-1}X$. Why is this all you need to do? Because this is precisely what is required to factor the Gaussian density $$ f(x, y) = |2\pi V|^{-1/2}\exp\left(-\frac 1 2 (x, y)^T V^{-1} (x, y)\right) $$ into a product of two Gaussians (one Gaussian representing the marginal of $X$ and the other the conditional distribution of $Y | X = x$). This, I think, just involves sticky algebra; one lesson here is that Schur complements play a nice algebraic role in the breaking up of quadratic forms that look like $z^T A^{-1}z$, but maybe someone can point out something deeper going on.

The second part of the question, as stated currently, doesn't make any sense; $V^{-1} = 0$ is an impossible equation to satisfy. Maybe what you mean is that you want to show $\mbox{Var}(X|Y) = 0$ provided that $V$ is singular? (This is false, by the way).

Answer 2

The detailed derivation may be found in:

von Mises, Richard (1964). Mathematical theory of probability and statistics. Chapter VIII.9.3. Academic Press.

I have added this reference to the Wikipedia article that you linked to, for the benefit of anyone else who stumbles over this question.