I'm currently working through the chapter on finite mixture models in BDA3 and came across the following model setup (with the usual slight abuse of notation):

Let $\lambda=(\lambda_1,\dots,\lambda_H)$, $\sum_{h=1}^H\lambda_h = 1$ be the mixing proportions and let $f(y_i\mid\theta_h)$ be some parametric density.

Then the observations $y=(y_1,\dots,y_n)$ can be modeled as a continuous real-valued random variable with density

$$p(y\mid \theta,\lambda) = \prod_{i=1}^n\sum_{h=1}^H \lambda_h f(y_i\mid\theta_h).$$

Alternatively we can introduce latent class indicators $z_{ih}$

$$z_{ih} = \begin{cases}1 & \text{if i-th observation is drawn from h-th mixture component} \\ 0 &\text{otherwise}\end{cases} $$

with distribution

$$z_i = (z_{i1},\dots,z_{iH}) \mid \lambda \sim \text{Multinomial}(1; \lambda)$$

and corresponding probability mass function $p(z_i\mid\lambda)$ which lets us write

$$p(y\mid z, \theta) = \prod_{i=1}^n \prod_{h=1}^H (f(y_i\mid\theta_h))^{z_{ih}}.$$

So far things seem clear. But then the authors write:

The joint distribution of the observed data $y$ and the unobserved indicators $z$ conditional on the model parameters can be written $$p(y, z\mid\theta,\lambda) = p(z\mid\lambda)p(y\mid z, \theta) = \prod_{i=1}^n\prod_{h=1}^H(\lambda_hf(y_i\mid\theta_h))^{z_{ih}}$$ with exactly one of $z_{ih}$ equaling $1$ for each $i$.

From the perspective of probability theory we can not just multiply $p(y\mid z,\theta)$ and $p(z\mid\lambda)$, as the former is the pdf of a continous random variable and the latter is the pmf of a discrete random variable. On the other hand, we can multiply with priors for $\lambda$ and $\theta$, work out the full conditionals and get a working Gibbs sampler.

So, my question is: What is $p(y, z\mid\theta,\lambda)$ as defined above, if it is neither a pdf nor pmf?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.