Can I say anything about the shape of the posterior distribution?

Question

I have a model where the noise is modelled as independent and identically distributed across the various data points. The noise $e$ is modelled as a 0 mean gaussian with $\phi$ as the precision (inverse variance). The likelihood for a single data point $i$ is defined as:

$$ P(y|w, \phi) = (\frac{\phi}{2\pi})^{0.5} \exp ^{-0.5 e_{i}\phi e_{i}} $$

The prior $P(w|y)$ is also modelled as a multivariate gaussian with zero mean and a covariance matrix. There is a scale parameter $\lambda$ which is also defined probabilistically and modelled using a gamma distribution. The noise precision $\phi$ also needs be inferred and is modelled using a gamma distribution

So ultimately the posterior will be given us

$$ P(w, \phi, \lambda|y) = \frac{P(w|\lambda) p(\phi) p(\lambda) \prod_{i} P(y|w, \phi)}{P(y)} $$

Now, this is probably a really silly question but since the priors are modelled using the Gamma or Gaussian distribution and the likelihood is also modelled with Gaussian noise, can we say that the posterior distribution will have also belong, say to the exponential family of distributions? I would like to know how the shape of priors and the likelihood affect the shape of the posterior distribution, if at all.

Answer 1

I would like to know how the shape of priors and the likelihood affect the shape of the posterior distribution, if at all.

The posterior is proportional to the product of the likelihood and the prior:

$$p(\theta |\textrm{Data})=\frac{p(\textrm{Data}|\theta)p(\theta)}{P(\textrm{Data})} $$ so

$$p(\theta |\textrm{Data}) \propto p(\textrm{Data}|\theta)p(\theta)$$

can we say that the posterior distribution will have also belong, say to the exponential family of distributions?

Your prior will be in the form:

$$p(\theta) = {(\frac{\phi}{2\pi}})^{0.5}\, e^{\frac{-\phi(\theta-\mu)^2}{2}} $$

which uses the parametrization of the Normal Distribution with precision $\phi$.

This, in turn, is in the same algebraic form as your Likelihood:

$$P(y|w, \phi) = (\frac{\phi}{2\pi})^{0.5} \exp ^{-0.5 e_{i}\phi e_{i}}$$

Therefore, per my answer to your first question, the posterior will be in this exponential form.

$$p(\theta |\textrm{Data}) \propto {(\frac{\phi}{2\pi}})^{0.5}\, e^{\frac{-\phi(\theta-\mu)^2}{2}} * (\frac{\phi}{2\pi})^{0.5} \exp ^{-0.5 e_{i}\phi e_{i}}$$

P.S: I realised the parameterisations in the formulae are incosistent, but got to run now. The statement on the algebraic form holds.