3
$\begingroup$

I have a joint distribution which can be written down as follows:

$$ p(\Theta | D) \propto p(D|\Theta) \; p(w|\lambda) \; p(\lambda) \; p(\phi) $$

where $\Theta = \{w, \lambda, \phi\}$. The prior distribution on $\lambda$ and $\phi$ are modelled using a Gamma distribution with some initial scale and shape parameters. The prior on $w$ is given by the term $p(w|\lambda)$ which is a multivariate normal with $0$ mean and some precision given by $\lambda \Sigma^{-1}$. The likelihood term is given by $p(D|\Theta)$.

Now, I would like to get the updates for the parameters for $\lambda$ and $\phi$, for a given $w$. So, for a given estimate of $w$, I would like to update the current posterior estimates of the Gamma distribution for $\lambda$ and $\phi$. Since, the Gamma prior is conjugate to the likelihood term which is modelled as normal, there should be some explicit updates to these parameters, I am guessing. However, I do not know much about this and do not know how to proceed. Also, the prior $p(w|\lambda)$ has this dependency on $\lambda$ and I am not sure if this is a problem.

Appreciate any suggestions on how to begin to proceed on this.

$\endgroup$
1
$\begingroup$

See the paper Inferring a Gaussian distribution or any textbook on Bayesian inference.

$\endgroup$
  • $\begingroup$ Still been having trouble figuring this out. I understand the updates for the prior parameters for the mean and precision of a Gaussian. However, here we have two parameters $\lambda$ which is the linear scaling factor of the precision matrix for the prior on 'w' and the global noise variance $\phi$ parameter. So, if I want to, for example, update the $\lambda$ parameter, should I try and integrate the $\phi$ parameter out from the expression of the posterior distribution before trying to update $\lambda$ and vice versa? $\endgroup$ – Luca Mar 10 '14 at 14:24
  • 2
    $\begingroup$ This depends on what kind of approximation you want for the posterior. If you want the marginal distribution of $\lambda$ then yes you need to integrate out the other parameters. $\endgroup$ – Tom Minka Mar 10 '14 at 14:36
  • $\begingroup$ Thanks. Yes, I wanted to update the parameters of the posterior distributions for $\lambda$ and $\phi$. $\endgroup$ – Luca Mar 10 '14 at 14:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.