Conditional expectation of random vector given low-rank linear transform

Question

Given the random vector $$ \mathbf{h} = \left(\begin{matrix} \mu \\ \varepsilon_1 \\ \varepsilon_2 \end{matrix} \right) \thicksim \mathcal{N}(\mathbf{0}, \mathbf{\Sigma}_\mathbf{h}) $$ and the low-rank transform: $$ \mathbf{P} = \left( \begin{matrix} 1 & 1 & 0 \\ 1 & 0 & 1 \\ \end{matrix} \right) $$ it's easy to show that $$ \mathbf{P}\mathbf{h} = \left( \begin{matrix}\mu + \varepsilon_1 \\ \mu + \varepsilon_2 \end{matrix} \right) = \left( \begin{matrix} x_1 \\ x_2 \end{matrix}\right) = \mathbf{x} \thicksim \mathcal{N}(\mathbf{0}, \mathbf{\Sigma_x}) $$ with $$ \mathbf{\Sigma_x} = \mathbf{P\Sigma_\mathbf{h}P}^T $$

My question is how to compute the expectation of $\mathbf{h}$ given $\mathbf{x}$. I'm lead to believe that

$$ \mathbb{E}[\mathbf{h}|\mathbf{x}] = \mathbf{\Sigma_h}\mathbf{P}^T\mathbf{\Sigma}_\mathbf{x}^{-1}\mathbf{x} $$

but I can't figure out how to derive that result. Any help would be much appreciated!

Answer 1

If we assume that $$ \mathbb{E}_\mathbf{h}[\mathbf{h}|\mathbf{x}] = \mathbf{Mx} $$ for some $\mathbf{M}$. Then $$ \begin{align} \mathrm{cov}(\mathbf{h}, \mathbf{x}) &= \mathrm{cov}(\mathbf{h}, \mathbf{x}) \\ \mathrm{cov}(\mathbf{h}, \mathbf{Ph}) &= \mathrm{cov}(\mathbb{E}_\mathbf{h}[\mathbf{h}|\mathbf{x}], \mathbf{x})\\ \mathrm{cov}(\mathbf{h}, \mathbf{h})\mathbf{P}^T &= \mathrm{cov}(\mathbf{Mx}, \mathbf{x}) \\ &= \mathbf{M}\mathrm{cov}(\mathbf{x}, \mathbf{x}) \\ \mathbf{\Sigma_h}\mathbf{P}^T &= \mathbf{M}\mathbf{\Sigma_x} \\ \mathbf{\Sigma_h}\mathbf{P}^T\mathbf{\Sigma}_\mathbf{x}^{-1} &= \mathbf{M} \\ \end{align} $$

The linearity assumption can be made because all distributions are Gaussian.