0
$\begingroup$

I am trying to derive a formula of a paper, and quite stuck at how they did that. Hopefully, someone can help me. So as we all know, for a $d$-dimensional random vector $\boldsymbol{X} \sim \mathcal{N} \left( \boldsymbol{\mu}, \boldsymbol{\Sigma} \right)$, its probability density function (pdf) is $$f_\text{MN} \left( \boldsymbol{x} ; \boldsymbol{\mu}, \boldsymbol{\Sigma} \right) = (2 \pi)^{-d / 2} | \boldsymbol{\Sigma} |^{-1/2} \exp \left \{ -\frac{1}{2} \left( \boldsymbol{x} - \boldsymbol{\mu} \right)^\top \boldsymbol{\Sigma}^{-1} \left( \boldsymbol{x} - \boldsymbol{\mu} \right) \right \}.$$

If $\boldsymbol{X}$ is decomposed into $\underset{p \times 1}{\boldsymbol{X}_1}$ and $\underset{q \times 1}{\boldsymbol{X}_2}$, where $p + q = d$, then $\begin{bmatrix} \boldsymbol{X}_1 \\ \boldsymbol{X}_2 \end{bmatrix} \sim \mathcal{N} \left( \begin{bmatrix} \boldsymbol{\mu}_1 \\ \boldsymbol{\mu}_2 \end{bmatrix}, \begin{bmatrix} \boldsymbol{\Sigma}_{11} & \boldsymbol{\Sigma}_{12} \\ \boldsymbol{\Sigma}_{21} & \boldsymbol{\Sigma}_{22} \end{bmatrix} \right)$, and the pdf becomes

$$f_\text{MN} \left( \boldsymbol{x}_1, \boldsymbol{x}_2 ; \boldsymbol{\mu}, \boldsymbol{\Sigma} \right) = (2 \pi)^{-(p + q) / 2} | \boldsymbol{\Sigma} |^{-1/2} \exp \left \{ -\frac{1}{2} \begin{bmatrix} \boldsymbol{x}_1 - \boldsymbol{\mu}_1 & \boldsymbol{x}_2 - \boldsymbol{\mu}_2 \end{bmatrix} \begin{bmatrix} \boldsymbol{\Sigma}_{11} & \boldsymbol{\Sigma}_{12} \\ \boldsymbol{\Sigma}_{21} & \boldsymbol{\Sigma}_{22} \end{bmatrix}^{-1} \begin{bmatrix} \boldsymbol{x}_1 - \boldsymbol{\mu}_1 \\ \boldsymbol{x}_2 - \boldsymbol{\mu}_2 \end{bmatrix} \right \}.$$ Now suppose that $\boldsymbol{X}_1 = \boldsymbol{x}_1$, meaning $\boldsymbol{X}_1$ is known, and we need to find $E \left \{ \ln f_\text{MN} \left( \boldsymbol{X}_1, \boldsymbol{X}_2 ; \boldsymbol{\mu}, \boldsymbol{\Sigma} \right) \middle| \boldsymbol{X}_1 = \boldsymbol{x}_1 \right \}$. They ended up having

$$E \left \{ \ln f_\text{MN} \left( \boldsymbol{X}_1, \boldsymbol{X}_2 ; \boldsymbol{\mu}, \boldsymbol{\Sigma} \right) \middle| \boldsymbol{X}_1 = \boldsymbol{x}_1 \right \} = -\frac{p}{2} \ln (2\pi) - \frac{1}{2} \ln | \boldsymbol{\Sigma} | - \frac{1}{2}\begin{bmatrix} \boldsymbol{x}_1 - \boldsymbol{\mu}_1 & \boldsymbol{x}_2 - \boldsymbol{\mu}_2 \end{bmatrix} \begin{bmatrix} \boldsymbol{\Sigma}_{11} & \boldsymbol{\Sigma}_{12} \\ \boldsymbol{\Sigma}_{21} & \boldsymbol{\Sigma}_{22} \end{bmatrix}^{-1} \begin{bmatrix} \boldsymbol{x}_1 - \boldsymbol{\mu}_1 \\ \boldsymbol{x}_2 - \boldsymbol{\mu}_2 \end{bmatrix}$$.

I just don't understand the $-\frac{p}{2}$. Should it be $-\frac{p + q}{2}$? Perhaps we are now conditioning on $\boldsymbol{X}_1 = \boldsymbol{x}_1$, so the dimension changes according to the dimension of $\boldsymbol{X}_1$, which is $p$. I don't know if that is the right explanation. Please help me with this if you can. Thank you so much.

$\endgroup$
1
  • $\begingroup$ This doesn't make sense. The expected value can depend on $x_1$ and all the parameters of the distribution, but it cannot depend on $x_2$. Furthermore, the density of the distribution of $X_2$ conditional on a fixed value of $X_1=x_1$ is multivariate normal with dimension $q$. So, if that's what it means, there should be $q$ in the numerator, not $p$. $\endgroup$
    – John L
    Jan 31, 2021 at 2:05

1 Answer 1

0
$\begingroup$

The distribution of $X_2$ conditional on $X_1=x_1$ is multivariate normal with mean $\mu_2+\Sigma_{21}\Sigma_{11}^{-1}(x_1-\mu_1)$ and covariance matrix $\Sigma_{22}-\Sigma_{21}\Sigma_{11}^{-1}\Sigma_{12}$.

The expected value of logarithm of the density for this random variable (see here) is
$$-\frac{1}2 \log\det\left((\Sigma_{22}-\Sigma_{21}\Sigma_{11}^{-1}\Sigma_{12})\right)-\frac{q}2 (1+\log(2\pi))$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.