# Dimension changes when taking conditional expectation

I am trying to derive a formula of a paper, and quite stuck at how they did that. Hopefully, someone can help me. So as we all know, for a $$d$$-dimensional random vector $$\boldsymbol{X} \sim \mathcal{N} \left( \boldsymbol{\mu}, \boldsymbol{\Sigma} \right)$$, its probability density function (pdf) is $$f_\text{MN} \left( \boldsymbol{x} ; \boldsymbol{\mu}, \boldsymbol{\Sigma} \right) = (2 \pi)^{-d / 2} | \boldsymbol{\Sigma} |^{-1/2} \exp \left \{ -\frac{1}{2} \left( \boldsymbol{x} - \boldsymbol{\mu} \right)^\top \boldsymbol{\Sigma}^{-1} \left( \boldsymbol{x} - \boldsymbol{\mu} \right) \right \}.$$

If $$\boldsymbol{X}$$ is decomposed into $$\underset{p \times 1}{\boldsymbol{X}_1}$$ and $$\underset{q \times 1}{\boldsymbol{X}_2}$$, where $$p + q = d$$, then $$\begin{bmatrix} \boldsymbol{X}_1 \\ \boldsymbol{X}_2 \end{bmatrix} \sim \mathcal{N} \left( \begin{bmatrix} \boldsymbol{\mu}_1 \\ \boldsymbol{\mu}_2 \end{bmatrix}, \begin{bmatrix} \boldsymbol{\Sigma}_{11} & \boldsymbol{\Sigma}_{12} \\ \boldsymbol{\Sigma}_{21} & \boldsymbol{\Sigma}_{22} \end{bmatrix} \right)$$, and the pdf becomes

$$f_\text{MN} \left( \boldsymbol{x}_1, \boldsymbol{x}_2 ; \boldsymbol{\mu}, \boldsymbol{\Sigma} \right) = (2 \pi)^{-(p + q) / 2} | \boldsymbol{\Sigma} |^{-1/2} \exp \left \{ -\frac{1}{2} \begin{bmatrix} \boldsymbol{x}_1 - \boldsymbol{\mu}_1 & \boldsymbol{x}_2 - \boldsymbol{\mu}_2 \end{bmatrix} \begin{bmatrix} \boldsymbol{\Sigma}_{11} & \boldsymbol{\Sigma}_{12} \\ \boldsymbol{\Sigma}_{21} & \boldsymbol{\Sigma}_{22} \end{bmatrix}^{-1} \begin{bmatrix} \boldsymbol{x}_1 - \boldsymbol{\mu}_1 \\ \boldsymbol{x}_2 - \boldsymbol{\mu}_2 \end{bmatrix} \right \}.$$ Now suppose that $$\boldsymbol{X}_1 = \boldsymbol{x}_1$$, meaning $$\boldsymbol{X}_1$$ is known, and we need to find $$E \left \{ \ln f_\text{MN} \left( \boldsymbol{X}_1, \boldsymbol{X}_2 ; \boldsymbol{\mu}, \boldsymbol{\Sigma} \right) \middle| \boldsymbol{X}_1 = \boldsymbol{x}_1 \right \}$$. They ended up having

$$E \left \{ \ln f_\text{MN} \left( \boldsymbol{X}_1, \boldsymbol{X}_2 ; \boldsymbol{\mu}, \boldsymbol{\Sigma} \right) \middle| \boldsymbol{X}_1 = \boldsymbol{x}_1 \right \} = -\frac{p}{2} \ln (2\pi) - \frac{1}{2} \ln | \boldsymbol{\Sigma} | - \frac{1}{2}\begin{bmatrix} \boldsymbol{x}_1 - \boldsymbol{\mu}_1 & \boldsymbol{x}_2 - \boldsymbol{\mu}_2 \end{bmatrix} \begin{bmatrix} \boldsymbol{\Sigma}_{11} & \boldsymbol{\Sigma}_{12} \\ \boldsymbol{\Sigma}_{21} & \boldsymbol{\Sigma}_{22} \end{bmatrix}^{-1} \begin{bmatrix} \boldsymbol{x}_1 - \boldsymbol{\mu}_1 \\ \boldsymbol{x}_2 - \boldsymbol{\mu}_2 \end{bmatrix}$$.

I just don't understand the $$-\frac{p}{2}$$. Should it be $$-\frac{p + q}{2}$$? Perhaps we are now conditioning on $$\boldsymbol{X}_1 = \boldsymbol{x}_1$$, so the dimension changes according to the dimension of $$\boldsymbol{X}_1$$, which is $$p$$. I don't know if that is the right explanation. Please help me with this if you can. Thank you so much.

• This doesn't make sense. The expected value can depend on $x_1$ and all the parameters of the distribution, but it cannot depend on $x_2$. Furthermore, the density of the distribution of $X_2$ conditional on a fixed value of $X_1=x_1$ is multivariate normal with dimension $q$. So, if that's what it means, there should be $q$ in the numerator, not $p$. Jan 31, 2021 at 2:05

The distribution of $$X_2$$ conditional on $$X_1=x_1$$ is multivariate normal with mean $$\mu_2+\Sigma_{21}\Sigma_{11}^{-1}(x_1-\mu_1)$$ and covariance matrix $$\Sigma_{22}-\Sigma_{21}\Sigma_{11}^{-1}\Sigma_{12}$$.

The expected value of logarithm of the density for this random variable (see here) is
$$-\frac{1}2 \log\det\left((\Sigma_{22}-\Sigma_{21}\Sigma_{11}^{-1}\Sigma_{12})\right)-\frac{q}2 (1+\log(2\pi))$$