2
$\begingroup$

For a normal likelihood $$ P(\mathbf{y}|\mathbf{b}) = \mathcal{N}(\mathbf{Gb}, \mathbf{\Sigma}_y) $$ and a normal prior $$ P(\mathbf{b}) = \mathcal{N}(\mathbf{\mu}_p, \mathbf{\Sigma}_p) $$ I'm trying derive the evidence (or marginal likelihood) $P(\mathbf{y})$ where $$ P(\mathbf{y}) = \int P(\mathbf{y, b}) \, \mathrm{d}\mathbf{b} = \int P(\mathbf{y|b}) P(\mathbf{b}) \, \mathrm{d}\mathbf{b} = \mathcal{N}(\mu_{\mathrm{ML}}, \mathbf{\Sigma}_{\mathrm{ML}}) $$ Could anyone point me at a source for this derivation (or reproduce it)?

I tried doing it in an indirect way by first finding the posterior $P(\mathbf{b|y})$ and then via Bayes' theorem writing $$ \ln{P(\mathbf{y})} = \ln{P(\mathbf{y|b})} + \ln{P(\mathbf{b})} - \ln{P(\mathbf{b|y})}, $$ then by grouping terms quadratic in $\mathbf{y}$ and using the Woodbury identity I managed to show that the marginal likelihood covariance is $\mathbf{\Sigma}_{\mathrm{ML}} = \mathbf{G} \mathbf{\Sigma}_p \mathbf{G}^{\top} + \mathbf{\Sigma}_y$, but I haven't been able to get the mean $\mu_{\mathrm{ML}}$.

I'm sure the integral can be evaluated directly to get the answer, but I'm not sure how to go about it.

Thanks!

$\endgroup$
4
$\begingroup$

The key here is to rewrite $y$ as $$y = Gb + e$$ where $G$ is known, $b\sim N(\mu_p,\Sigma_p)$ and independently $e\sim N(0,\Sigma_y)$. Now $y$ is just the sum of two independent normals which is fully determined by its mean and covariance matrix. The mean is $$E[Y] = E[Gb] + E[e] = GE[b] + 0 = G\mu_p$$ and the variance is $$Var[y] = Var[Gb] + Var[e] = GVar[b]G^\top + Var[e] = G\sigma_pG^\top +\Sigma_y$$ since $b$ and $e$ are independent the covariance is zero. Thus the marginal likelihood is $$y\sim N(G\mu_p,G\Sigma_pG^\top + \Sigma_y).$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.