Mutual information between subsets of variables in the multivariate normal distribution Let $\vec{X}$ be a random vector following a multi-variate normal distribution $P(\vec X)$ with covariance matrix $\Sigma$ and zero means  (for simplicity). Consider a partition of $\vec X$ into two subsets of variables, $\vec X = \{\vec X_1, \vec X_2\}$. What is the mutual information between $\vec X_1$ and $\vec X_2$:
$$I(\vec X_1; \vec X_2) = \int P(\vec X)\ln \frac{P(\vec X)}{P_1(\vec X_1)P_2(\vec X_2)} \mathrm{d}\vec X$$
where
$$P_1(\vec X_1) = \int P(\vec X)\mathrm{d}\vec X_2, \qquad
  P_2(\vec X_2) = \int P(\vec X)\mathrm{d}\vec X_1  $$
I presume an analytical answer can be given, but I've been having trouble obtaining it.
 A: Actually the answer is trivial to obtain.
$$I(\vec X_1,\vec X_2) = S_1 + S_2 - S_X$$
where $S_1$ is the entropy of $\vec X_1$, $S_2$ the entropy of $\vec X_2$, and $S_X$ the entropy of the full distribution of $\vec X$.
Therefore we obtain the result
$$I(\vec X_1,\vec X_2) = \frac{1}{2} \ln \left( \frac{\det (\Sigma_1) \det (\Sigma_2)}{\det (\Sigma)}
\right)$$
where we write the covarinace matrix in the block-form:
$$\Sigma = \left(\begin{array}{cc}
  \Sigma_{11} & \Sigma_{12}\\
  \Sigma_{21} & \Sigma_{22}
\end{array}\right)$$
A: To see where the results comes from using another approach, first recall that if $X,Y$ have joint distribution $p(X,Y)$ and marginals $p(X)$ and $p(Y)$, respectively, then the mutual information is the Kullback-Leibler (KL) divergence between the joint distribution and product of marginal distributions:
$$I(X:Y) = D_{\rm KL}(p(X,Y)\mid\mid p(X)p(Y)).$$
Next, for a pair of $n$-dimensional multivariate normal distributions
with mean and covariance parameters $(\mu, \Sigma)$ and $(\tilde{\mu}, \tilde{\Sigma})$, respectively, the KL divergence is given by:
$$
\begin{aligned}
&D_{\rm KL}((\mu,\Sigma) \mid\mid (\tilde{\mu}, \tilde{\Sigma})) \\
&= \frac{1}{2} \left[
    \log\left( \mathrm{det}(\tilde{\Sigma})/ \mathrm{det}(\Sigma) \right)
    + \mathrm{tr}\left( \Sigma \tilde{\Sigma}^{-1} \right) - n
    + (\mu - \tilde{\mu})^\top\tilde{\Sigma}^{-1}(\mu - \tilde{\mu})
\right].
\end{aligned}
$$
Suppose now that the joint distribution $p(X,Y)$ is multivariate normal with mean and covariance given by
$$
\mu = \begin{bmatrix} \mu_x \\ \mu_y \end{bmatrix}, \qquad 
\Sigma = \begin{bmatrix} \Sigma_x & \Sigma' \\ \Sigma' & \Sigma_y \end{bmatrix}.
$$
The product of marginals $p(X)p(Y)$ is also multivariate normal, with mean and covariance given by
$$
\tilde{\mu} = \begin{bmatrix} \mu_x \\ \mu_y \end{bmatrix}, \qquad 
\tilde{\Sigma} = \begin{bmatrix} \Sigma_x & \mathbf{0} \\ \mathbf{0} & \Sigma_y \end{bmatrix}.
$$
To complete the computation, we simplify the expression for $D_{\rm KL}$ between two multivariate normal distributions whose are parameters are $(\mu, \Sigma)$ and $(\tilde{\mu}, \tilde{\Sigma})$ as given above, by using the following facts:

*

*Since $\tilde{\Sigma}$ is a block matrix and $\Sigma_y$ is invertible, we have

$$\mathrm{det}(\tilde{\Sigma}) = \mathrm{det}(\Sigma_x - \mathbf{0}\Sigma_y^{-1}\mathbf{0})\mathrm{det}(\Sigma_y) =
\mathrm{det}(\Sigma_x)(\Sigma_y).
$$


*From the block structure of $\tilde{\Sigma}$, we have

$$
\tilde{\Sigma}^{-1} =
\begin{bmatrix} \Sigma_x^{-1} & \mathbf{0} \\ \mathbf{0} & \Sigma_y^{-1} \end{bmatrix}
$$
which implies that
$$
\Sigma \tilde{\Sigma}^{-1} = 
\begin{bmatrix} \Sigma_x & \Sigma' \\ \Sigma' & \Sigma_y \end{bmatrix}
\begin{bmatrix} \Sigma_x^{-1} & \mathbf{0} \\ \mathbf{0} & \Sigma_y^{-1} \end{bmatrix}
=
\begin{bmatrix} \Sigma_x\Sigma_x^{-1} & \Sigma'\Sigma_y \\ \Sigma'\Sigma_x & \Sigma_y^{-1} \end{bmatrix}
=
\begin{bmatrix} \mathbf{1}_{n_1} & \Sigma'\Sigma_y \\ \Sigma'\Sigma_x & \mathbf{1}_{n_2} \end{bmatrix}
$$
where $n_1$ and $n_2$ are positive integers such that $n_1 + n_2 = n$. Thus, it follows that
$$
\mathrm{tr}\left( \Sigma \tilde{\Sigma}^{-1} \right) - n = n - n = 0.
$$


*Since $\mu = \tilde{\mu}$, the right-most term in the expression for $D_{\rm KL}$ is equal to zero:

$$
\frac{1}{2}(\mu - \tilde{\mu})^\top\tilde{\Sigma}^{-1}(\mu - \tilde{\mu}) = 0.
$$
Using these facts, we have shown that
$$
I(X:Y) = D_{\rm KL}((\mu,\Sigma) \mid\mid (\tilde{\mu}, \tilde{\Sigma})) = 
\frac{1}{2} \left[
    \log\left(
    \frac{\mathrm{det}(\Sigma_x)\mathrm{det}(\Sigma_y)}{\mathrm{det}(\Sigma)} 
    \right)
\right]
$$
as desired.
