2
$\begingroup$

Suppose we have a joint distribution on vector $[\mathbf{x}, y]$: $$ p([y, \mathbf{x}] ) = \mathcal{N}\left(\begin{pmatrix} y \\ \mathbf{x}\end{pmatrix}| 0, \begin{pmatrix} k& \mathbf{v} \\ \mathbf{v}^T & K\end{pmatrix}\right), $$ where $\mathbf{x} \in \mathbb{R}^N$, $y \in \mathbb{R}$. And also we know distribution of $\mathbf{x}$ conditioned on some data $D$: $$ q(\mathbf{x}| D) = \mathcal{N} (\mu, \sigma^2 I). $$ How does analytical expression for $p(y| D)$ look like (i.e. how to handle such an integral in the simplest way)?: $$ p(y| D) = \int p(y| \mathbf{x}) q(\mathbf{x}| D) d \mathbf{x} = ? $$

So, I try to solve this, but all I obtain looks like a monster, which isn't appropriate for me, so I have to use this expression further through my research.

$\endgroup$
3
  • 1
    $\begingroup$ The difficulty with Bayesian methods when the prior is not nice (like a conjugate prior) is that the normalization constant is an integral that can be difficult to evaluate. This is one of the reason MCMC methods are so popular. $\endgroup$ Jun 14 '12 at 15:22
  • $\begingroup$ Would solving up to proportionality be permitted, or is this one of the cases where you need to explicitly calculate the normalization constant? $\endgroup$ Jun 14 '12 at 15:24
  • 1
    $\begingroup$ The posterior $p(y| D)$ is seems to be normal, so all I need is mean and variance. $\endgroup$ Jun 14 '12 at 15:27
2
$\begingroup$

It is much easier to do these evidence combination operations in the natural parametrization, which for the multivariate normal distribution with mean $\mu$ and covariance matrix $\Sigma$ is $$\begin{bmatrix}\Sigma^{-1}\mu \\ \Sigma^{-1}\end{bmatrix}.$$

Convert $p$ and $q$ so that they're in this parametrization, then pad $q$ with zeroes for the extra component (since the precision there is zero) and just add the natural parameters.

Therefore, the resulting distribution is: \begin{align} y \mid D \sim \mathcal N\left([\Psi\Phi\mu]_1, \Psi_{11}\right) \end{align} where \begin{align} \Theta &= \begin{bmatrix} k& \mathbf{v} \\ \mathbf{v}^T & K\end{bmatrix}^{-1} \\ \Phi &= \begin{bmatrix} 0& \mathbf{0} \\ \mathbf{0}^T & \sigma^{-2}I\end{bmatrix} \\ \Psi &=(\Theta + \Phi)^{-1}. \end{align}

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.