I want to understand about Bayesian mixture model discussed in RJMCMC paper (Richardson and Green, 1997) (https://academic.oup.com/jrsssb/article/59/4/731/7083042) I also posted similar question earlier but with not clarifying my point. enter image description here

enter image description here

Then here is my question regarding that setting. In the text "In this situation, it is natural to regard the group label z_i for the i th observation as a latent allocation variable. The z_i are supposed independently drawn from the distribution"

  1. Then, how can we understand the meaning of z_i in this context? by Equation (2) p(z_i=j)=w_j, It also implies p(z_i~=j)=0?

  2. In the assumption of model, the observation y_j is determined by w_j*f(.|θ_j) (Eq (1)). Therefore, isn't y_j should be determinisitic by θ,z,w,k? Then, why we have to consider about the prior of p(y|θ,z,w,k) if it is explicitly determined by θ,z,w,k.

Thanks for reading my questions. Hope my questions are clarified well in this time.

  • $\begingroup$ This is not a question related to RJMCMC so you should remove the term from the title. $\endgroup$
    – Xi'an
    Aug 29 at 7:11

1 Answer 1


When considering a random variable distributed from a mixture model $$Y\sim\sum_{j=1}^k w_j f(y|\theta_j)\tag{1}$$ this random variable can be expressed as the marginal of a pair $(Y,Z)$ of random variables when their joint distribution is defined as $$Z\sim \mathbb P(Z=j)=w_j,\quad Y|Z\sim f(y|\theta_Z)$$ for $j=1,\ldots,k$ (the possible values of $Z$). This can easily be shown as $$\mathbb P(Y\in A)=\mathbb E^Z[\mathbb P(Y\in A|Z)]=\mathbb E^Z\left[\int_Af(y|\theta_Z)\,\text dy\right]=\sum_{j=1}^k w_j \int_Af(y|\theta_j)\,\text dy$$ for every measurable set $A$. In generative model terms, this means that $Y$ can be generated in two steps:

  1. Generate $U\sim\mathcal U(0,1)$ and set $$Z=\begin{cases} 1 &\text{when }U\le w_1\\ 2 &\text{when } w_1<U\le w_1+w_2\\\ & \qquad\vdots \\ k &\text{otherwise} \end{cases}$$
  2. Generate$$Y\sim f(y|\theta_Z)$$

and this leads to the most standard algorithm for generating realisations from (1).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.