Why does a zero entry in the inverse covariance matrix of a joint Gaussian distribution imply conditional independence?

Question

When $X_1, X_2, \ldots, X_n$ are random variables jointly following a Gaussian distribution, let $A$ be the inverse of the covariance matrix ($A=\Sigma^{-1}$). I am wondering how to prove following theorem:

If $A_{ij}=0,$ then $X_i$ and $X_j$ are conditionally independent on the other variables.

Answer 1

The aim of this answer is to demonstrate this result with minimal algebraic effort. The secret is to maintain a laser-like focus on what matters, ignoring all the rest. Let me illustrate.

By definition (many definitions anyway), the value of the density of a multivariate Gaussian with vector mean $\mu$ and invertible covariance matrix $\Sigma$ at the point $\mathbf x = (x_1, x_2, \ldots, x_n)^\prime$ is proportional to

$$\exp\left((\mathbf x - \mu)^\prime \Sigma^{-1} (\mathbf x - \mu)/2\right).$$

Conditioning on all the variables except $(x_i,x_j)$ is tantamount to viewing all those other variables as constants. Focus, then, on how this density depends on $(x_i,x_j).$ To do so, we must examine the argument of $\exp.$ Since $\mu$ is constant, too, let's consider how it depends on $(x_i-\mu_i,x_j-\mu_j) = (y_i,y_j).$ Letting $(a_{rs}),$ $1\le r,$ $1\le s$ be the coefficients of $\Sigma^{-1},$ the rules of matrix multiplication imply

$$\begin{aligned} (\mathbf x - \mu)^\prime \Sigma^{-1} (\mathbf x - \mu) &= a_{ii}y_i^2/2 + \text{constants}\times y_i \\&+ a_{jj}y_j^2/2 + \text{other constants}\times y_j \\ &+ a_{ij}y_iy_j \\&+ \text{yet other constants}. \end{aligned}$$

That's the sum of four expressions, one per line. The rules of exponentiation then tell us the conditional density is the product of five terms:

1. A normalizing constant from the original (joint) density.

2. A term that is a function only of $y_i$ -- which implies it's a function only of $x_i.$

3. Another term that is a function only of $y_j$ -- which implies it's a function only of $x_j.$

4. The $a_{ij}y_iy_j$ term.

5. The exponential of yet other constants.

When, as assumed in the question, $a_{ij} = 0,$ term (4) drops out. This leaves the product of constants (from $(1)$ and $(5)$), a function of $x_i$ alone (from $(2)$), and a function of $x_j$ alone (from $(3)$), showing explicitly how the conditional density factors as separate functions of $x_i$ and $x_j.$ This factorization implies the corresponding random variables are independent, QED.