# How to calculate a posterior for the given model?

Suppose we have a joint distribution on vector $[\mathbf{x}, y]$: $$p([y, \mathbf{x}] ) = \mathcal{N}\left(\begin{pmatrix} y \\ \mathbf{x}\end{pmatrix}| 0, \begin{pmatrix} k& \mathbf{v} \\ \mathbf{v}^T & K\end{pmatrix}\right),$$ where $\mathbf{x} \in \mathbb{R}^N$, $y \in \mathbb{R}$. And also we know distribution of $\mathbf{x}$ conditioned on some data $D$: $$q(\mathbf{x}| D) = \mathcal{N} (\mu, \sigma^2 I).$$ How does analytical expression for $p(y| D)$ look like (i.e. how to handle such an integral in the simplest way)?: $$p(y| D) = \int p(y| \mathbf{x}) q(\mathbf{x}| D) d \mathbf{x} = ?$$

So, I try to solve this, but all I obtain looks like a monster, which isn't appropriate for me, so I have to use this expression further through my research.

-
The difficulty with Bayesian methods when the prior is not nice (like a conjugate prior) is that the normalization constant is an integral that can be difficult to evaluate. This is one of the reason MCMC methods are so popular. –  Michael Chernick Jun 14 '12 at 15:22
Would solving up to proportionality be permitted, or is this one of the cases where you need to explicitly calculate the normalization constant? –  Jonathan Thiele Jun 14 '12 at 15:24
The posterior $p(y| D)$ is seems to be normal, so all I need is mean and variance. –  Alexey Jun 14 '12 at 15:27
add comment

## 1 Answer

It is much easier to do these evidence combination operations in the natural parametrization, which for the multivariate normal distribution with mean $\mu$ and covariance matrix $\Sigma$ is $$\begin{bmatrix}\Sigma^{-1}\mu \\ \Sigma^{-1}\end{bmatrix}.$$

Convert $p$ and $q$ so that they're in this parametrization, then pad $q$ with zeroes for the extra component (since the precision there is zero) and just add the natural parameters.

Therefore, the resulting distribution is: \begin{align} y \mid D \sim \mathcal N\left([\Psi\Phi\mu]_1, \Psi_{11}\right) \end{align} where \begin{align} \Theta &= \begin{bmatrix} k& \mathbf{v} \\ \mathbf{v}^T & K\end{bmatrix}^{-1} \\ \Phi &= \begin{bmatrix} 0& \mathbf{0} \\ \mathbf{0}^T & \sigma^{-2}I\end{bmatrix} \\ \Psi &=(\Theta + \Phi)^{-1}. \end{align}

-
add comment