Suppose you have vectors X and Y with covariance matrix $V = \left( \begin{array}{cc} A & B \\ B^T & C \end{array} \right)$. This Wikipedia article says that $Var(X | Y) = A - BC^{-1}B^T$, the Schur Complement of C in V.

Furthermore, conditional (in)dependence between X and Y can be found in the inverse of the covariance matrix $V^{-1}$. Earlier in the Wiki page they show that the inverse of a matrix is a function of the Schur Complement.

I'm trying to piece these two pieces together. First, how do we show $Var(X | Y) = A - BC^{-1}B^T$, and second: if $V^{-1} = 0$, how do we show that the Schur complement is necessarily 0, and hence $Var(X | Y) = 0$?

  • 4
    $\begingroup$ Please note that the setting is that of a multivariate normal distribution: otherwise, your assertions about (in)dependence are not true. $\endgroup$ – whuber Dec 22 '12 at 18:33

Assume WLOG that everything is mean $0$. If you know the formula for the inverse of a matrix in block form, then it should be as simple as checking that

$$(X, Y)^T V^{-1} (X, Y) = X^T A^{-1} X + (Y - \mu_{Y | X})^T S^{-1} (Y - \mu_{Y|X}) + c$$

where $S$ is the Schur complement and $c$ is a constant and $\mu_{Y|X} = B^TA^{-1}X$. Why is this all you need to do? Because this is precisely what is required to factor the Gaussian density $$ f(x, y) = |2\pi V|^{-1/2}\exp\left(-\frac 1 2 (x, y)^T V^{-1} (x, y)\right) $$ into a product of two Gaussians (one Gaussian representing the marginal of $X$ and the other the conditional distribution of $Y | X = x$). This, I think, just involves sticky algebra; one lesson here is that Schur complements play a nice algebraic role in the breaking up of quadratic forms that look like $z^T A^{-1}z$, but maybe someone can point out something deeper going on.

The second part of the question, as stated currently, doesn't make any sense; $V^{-1} = 0$ is an impossible equation to satisfy. Maybe what you mean is that you want to show $\mbox{Var}(X|Y) = 0$ provided that $V$ is singular? (This is false, by the way).


The detailed derivation may be found in:

von Mises, Richard (1964). Mathematical theory of probability and statistics. Chapter VIII.9.3. Academic Press.

I have added this reference to the Wikipedia article that you linked to, for the benefit of anyone else who stumbles over this question.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.