9
$\begingroup$

When $X_1, X_2, \ldots, X_n$ are random variables jointly following a Gaussian distribution, let $A$ be the inverse of the covariance matrix ($A=\Sigma^{-1}$). I am wondering how to prove following theorem:

If $A_{ij}=0,$ then $X_i$ and $X_j$ are conditionally independent on the other variables.

$\endgroup$
0

1 Answer 1

8
$\begingroup$

The aim of this answer is to demonstrate this result with minimal algebraic effort. The secret is to maintain a laser-like focus on what matters, ignoring all the rest. Let me illustrate.

By definition (many definitions anyway), the value of the density of a multivariate Gaussian with vector mean $\mu$ and invertible covariance matrix $\Sigma$ at the point $\mathbf x = (x_1, x_2, \ldots, x_n)^\prime$ is proportional to

$$\exp\left((\mathbf x - \mu)^\prime \Sigma^{-1} (\mathbf x - \mu)/2\right).$$

Conditioning on all the variables except $(x_i,x_j)$ is tantamount to viewing all those other variables as constants. Focus, then, on how this density depends on $(x_i,x_j).$ To do so, we must examine the argument of $\exp.$ Since $\mu$ is constant, too, let's consider how it depends on $(x_i-\mu_i,x_j-\mu_j) = (y_i,y_j).$ Letting $(a_{rs}),$ $1\le r,$ $1\le s$ be the coefficients of $\Sigma^{-1},$ the rules of matrix multiplication imply

$$\begin{aligned} (\mathbf x - \mu)^\prime \Sigma^{-1} (\mathbf x - \mu) &= a_{ii}y_i^2/2 + \text{constants}\times y_i \\&+ a_{jj}y_j^2/2 + \text{other constants}\times y_j \\ &+ a_{ij}y_iy_j \\&+ \text{yet other constants}. \end{aligned}$$

That's the sum of four expressions, one per line. The rules of exponentiation then tell us the conditional density is the product of five terms:

  1. A normalizing constant from the original (joint) density.

  2. A term that is a function only of $y_i$ -- which implies it's a function only of $x_i.$

  3. Another term that is a function only of $y_j$ -- which implies it's a function only of $x_j.$

  4. The $a_{ij}y_iy_j$ term.

  5. The exponential of yet other constants.

When, as assumed in the question, $a_{ij} = 0,$ term (4) drops out. This leaves the product of constants (from $(1)$ and $(5)$), a function of $x_i$ alone (from $(2)$), and a function of $x_j$ alone (from $(3)$), showing explicitly how the conditional density factors as separate functions of $x_i$ and $x_j.$ This factorization implies the corresponding random variables are independent, QED.

$\endgroup$
1
  • $\begingroup$ Thank you very much! $\endgroup$
    – gaussian
    Jun 8, 2022 at 5:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.