# Will this have a closed form expression: Conjugate priors

This is continuing the discussion about the posterior probability densities and conjugate priors.

So, my posterior distribution over the parameters is defined as:

$$P(w, \phi, \lambda | y) \propto P(\phi) P(w|\lambda) P(\lambda) \prod_{i}P(y|w, \phi, \lambda)$$

Now, $\phi$ is the parameter that models the noise precision. Noise is assumed to be global zero mean and IID distributed. So, the mean is known and we have an unknown precision and the conjugate prior is given by a Gamma distribution. So, $P(\phi) = Ga(\phi; a, b)$.

Now, the distribution over $w$ where both the mean and precision are unknown, we can write $P(w, \lambda)$ as $P(w|\lambda)P(\lambda)$. As described in Bishop's book, to achieve conjugacy, $P(w|\lambda)$ is modelled as a zero mean MVN whose precision is a function of $\lambda$ given by $(\lambda \Lambda)^{-1}$. Also, $P(\lambda)$ is modelled using a gamma distribution i.e. $P(\lambda) = Ga(\lambda; c, d)$.

So, now my questions are about what the posterior distribution.

1: Due to the use of such conjugate priors, is the posterior over all the parameters $P(w, \phi, \lambda | y)$ easier to $\textit{sample}$ from or is there a closed form expression for it which obviates the need for sampling all together and I can estimate the sufficient statistics analytically?

2: Can I say something that this joint posterior will have some particular form or can I only say something about the marginal distributions i.e. $P(w)$ will be normally distributed posteriorly, $P(\phi)$ will be gamma distributed etc. What about the sufficient statistics of these marginals? Can they be computed analytically? What about dependence/coupling of the parameters in the posterior distribution? I am guessing since we have coupling of $\lambda$ and $w$ in our model, it should be coupled in the posterior distribution as well. However, to perform any sort of approximate inference on such a system, do we $\textit{have}$ to using mean field approximations and assume independence between the parameters in the posterior distribution?

I missed a very fundamental point. The prior and the likelihood are not defined over the same parameters. In the example, I gave the prior is given by $P(w)$ and the likelihood is related to $w$ in a more complex and non-linear way. So, the likelihood is normal with respect to residuals (equivalent to minimising sum of square difference). So, they are normal but over completely different parameters and hence the posterior distribution will not be normal. At least this how I understand it after giving it much thought.