0
$\begingroup$

Let $X$ be a $D$-dimensional Gaussian random variable with distribution $N_D(x|\mu, \Sigma)$ in which the covariance $\Sigma$ is known. Given a vector of observations $x = (x_1,\ldots,x_n)$, we wish to infer the mean $\mu$.

Given a prior distribution $p(\mu) = N_D(\mu| \mu_0, \Sigma_0)$, find the corresponding posterior distribution $p(\mu|x)$.

I want to solve the problem above but I don't know how. Since I should get the $p(x|\mu)$ to find the posterior distribution and I don't have $\mu$, should I try maximum likelihood first?

$\endgroup$

1 Answer 1

0
$\begingroup$

First, apply Bayes Rule: $p(\mu|X)=\frac{p(X|\mu)p(\mu)}{p(X)}$. In the numerator, $p(X|\mu)$ and $p(\mu)$ are already given. Normally, you should calculate the denominator as $\int{p(X|\mu)p(\mu)d\mu}$. This integral might be cumbersome and sometimes can be avoided, i.e. when the format of the numerator is known, $p(X)$ can easily be guessed thanks to normalization. So, we have:

$$p(\mu|X)\propto p(X|\mu)p(\mu)\propto\exp(-\frac{1}{2}(x-\mu)^T\Sigma^{-1}(x-\mu))\exp(-\frac{1}{2}(\mu-\mu_0)^T\Sigma_0^{-1}(\mu-\mu_0))$$

In this expression, terms without $\mu$ can be treated as constants. So, $$p(\mu|X)\propto \exp((x^T\Sigma^{-1}+\mu_0^T\Sigma_0^{-1})\mu-\frac{1}{2}\mu^T(\Sigma^{-1}+\Sigma_0^{-1})\mu)\propto\exp(-\frac{1}{2}(\mu-a)^T\Sigma_\mu^{-1}(\mu-a))$$

which is in normal form. So, we actually found the posterior, just need to calculate the norm. constant wrt these parameters.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.