# Updating the hyper parameters for conjugate distributions

I have a joint distribution which can be written down as follows:

$$p(\Theta | D) \propto p(D|\Theta) \; p(w|\lambda) \; p(\lambda) \; p(\phi)$$

where $\Theta = \{w, \lambda, \phi\}$. The prior distribution on $\lambda$ and $\phi$ are modelled using a Gamma distribution with some initial scale and shape parameters. The prior on $w$ is given by the term $p(w|\lambda)$ which is a multivariate normal with $0$ mean and some precision given by $\lambda \Sigma^{-1}$. The likelihood term is given by $p(D|\Theta)$.

Now, I would like to get the updates for the parameters for $\lambda$ and $\phi$, for a given $w$. So, for a given estimate of $w$, I would like to update the current posterior estimates of the Gamma distribution for $\lambda$ and $\phi$. Since, the Gamma prior is conjugate to the likelihood term which is modelled as normal, there should be some explicit updates to these parameters, I am guessing. However, I do not know much about this and do not know how to proceed. Also, the prior $p(w|\lambda)$ has this dependency on $\lambda$ and I am not sure if this is a problem.

Appreciate any suggestions on how to begin to proceed on this.

• Still been having trouble figuring this out. I understand the updates for the prior parameters for the mean and precision of a Gaussian. However, here we have two parameters $\lambda$ which is the linear scaling factor of the precision matrix for the prior on 'w' and the global noise variance $\phi$ parameter. So, if I want to, for example, update the $\lambda$ parameter, should I try and integrate the $\phi$ parameter out from the expression of the posterior distribution before trying to update $\lambda$ and vice versa? – Luca Mar 10 '14 at 14:24
• This depends on what kind of approximation you want for the posterior. If you want the marginal distribution of $\lambda$ then yes you need to integrate out the other parameters. – Tom Minka Mar 10 '14 at 14:36
• Thanks. Yes, I wanted to update the parameters of the posterior distributions for $\lambda$ and $\phi$. – Luca Mar 10 '14 at 14:45