1
$\begingroup$

Why the expected value $\int h(x)f(x)$ is desired for inference in Dirichlet Process Mixture? What is the intuition for MCMC in Dirichlet Process Mixture?

$f(x)$ is the probability density function, and $h(x)$ is the outcome of the Dirichlet Process Mixture (I am not sure).

This formula is originally from page 40 of slides.

$\endgroup$
1
  • 1
    $\begingroup$ What are $h(x)$ and $f(x)$. Please add details $\endgroup$
    – niandra82
    Aug 14, 2014 at 12:49

1 Answer 1

1
$\begingroup$

Why the expected value ∫h(x)f(x) is desired for inference in Dirichlet Process Mixture?

I assume the author's $h(\cdot)$ is an arbitrary function of $x$. Monte Carlo estimation allows us to take expected values of arbitrary functions, where the expected value is over the posterior distribution $f(\cdot)$ in the above. So slide 40 is saying something very general about Bayesian stats, not particular information about DP models.

What is the intuition for MCMC in Dirichlet Process Mixture?

The DP mixture model is over the space of all partitions of the data. That's a pretty big/difficult thing to work with. MCMC for DP-mixtures works by moving around between plausible partitions of the dataset.

Inference for DP mixtures is still an active research topic because on big datasets the number of partitions is vast meaning finding 'good' ones is hard.

--EDIT: There are multiple Gibbs and other MCMC algorithms for the DP-MM. Neal 2000 gives a review as they stood then. For example, Algo 3 from that paper updates the latent class memberships one at a time, using conditional probability given all the other class assignments.--

Checking out Teh's Dirichlet Process entry for the encyclopedia of machine learning might help.

$\endgroup$
2
  • $\begingroup$ Thank you. As for as I know, Gibbs sampling is to use some tractable conditional probability to generate a sequence of samples, and then use these samples to estimate some features of the distribution, such as mean and variance. In the slides, from page 40 to 48, they are taking about inference in Dirichlet process mixture, especially on page 42 (Gibbs sampling), I do not know how make the connection between inference with Gibbs sampling. $\endgroup$ Aug 14, 2014 at 16:22
  • $\begingroup$ Updated with a link. Feel free to accept if this helps. $\endgroup$ Aug 14, 2014 at 17:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.