I have the following hierachical bayesian model -
$\mathbf{x}|\mathbf{c},\sigma^2 \sim \mathcal{N}(\mathbf{x}|\mathbf{c},\sigma^2)$
$\mathbf{c}|\mathbf{c}_1,\sigma^2_2 \sim \mathcal{N}(\mathbf{c}|\mathbf{c}_1,\sigma^2_2)$
$\sigma^2 \sim \text{InvGamma}(\alpha, \beta)$.

Here, $\mathbf{x}$ is a data sample. $\alpha$, $\beta$, $c_1$ and $\sigma_2^2$ are fixed known values. $\text{InvGamma}(\cdot)$ and $\mathcal{N}(\cdot)$ are inverse gamma and normal distributions respectively. The goal is to find the mode of the posterior distribution $p(\mathbf{c},\sigma^2|\mathbf{x})$. The method that I tried is Gibbs sampling to generate samples of $(\mathbf{c},\sigma^2)$ and then finding the mode of these samples using mean-shift algorithm. However, the method is computationally demanding and I wanted to know any alternative approaches to obtain the posterior mode efficiently.

  • $\begingroup$ Have you tried deriving the functional form of the posterior and working directly with that? $\endgroup$ – jbowman May 16 at 17:28
  • $\begingroup$ You mean taking the gradient of the function with respect to the variables and setting them to 0 ? $\endgroup$ – Buna May 16 at 18:34
  • $\begingroup$ In this case, the functional form of the posterior distribution should be easily recognizable, and finding the posterior mode is a trivial next step. $\endgroup$ – jbowman May 16 at 18:37
  • $\begingroup$ You mean it is trivial both for the mean and the variance ? $\endgroup$ – Buna May 16 at 18:49
  • $\begingroup$ Yes. But even if you don't recognize the functional form, it is still the case that the relevant function is continuous with first and second (and more) derivatives everywhere and unimodal, so many algorithms can find the mode to a high degree of accuracy. $\endgroup$ – jbowman May 16 at 18:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.