0
$\begingroup$

I am working on implementing the Bayesian hierarchical clustering algorithm found here from scratch as a way to practice and learn the algorithm. However, I have hit a snag in calculating the quantity in formula 1 of section 2:

\begin{align} p(D_k|H_1^k) &= \int p(D_k|\theta) p(\theta |\beta) d\theta\\ &= \int \left[ \prod_{x_i \in D_k} p(x_i |\theta) \right] p(\theta | \beta) d\theta \end{align}

In the subsequent paragraph, the authors state,

This is a natural model-based criterion for measuring how well the data fit into one cluster. If we choose models with conjugate priors (e.g. Normal-Inverse-Wishart priors for Normal continuous data or Dirichlet priors for Multinomial discrete data) this integral is tractable

I understand that this is the probability of cluster $k$ being generated from some distribution with parameter(s) $\mathbf{\theta}$ and that, on the first iteration of the algorithm, I should be calculating this quantity for every pair of data points (i.e. vectors) in the set.

However, I am a bit confused regarding the quote above. Does this mean that I should replace the contents of the integral with an appropriate posterior distribution and then perform the integration with respect to $\mathbf{\theta}$?

What would this mean in the Gaussian example mentioned by the authors:

For example, in the case of Gaussians, $p(D_k|H_1^k)$ is a function of the sample mean and covariance of the data in $D_k$

The paper seems to allude to this being a straight-forward calculation, but I have not seen conjugate priors used in this way. Any help would be appreciated!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.