1
$\begingroup$

I'm currently working through the chapter on finite mixture models in BDA3 and came across the following model setup (with the usual slight abuse of notation):

Let $\lambda=(\lambda_1,\dots,\lambda_H)$, $\sum_{h=1}^H\lambda_h = 1$ be the mixing proportions and let $f(y_i\mid\theta_h)$ be some parametric density.

Then the observations $y=(y_1,\dots,y_n)$ can be modeled as a continuous real-valued random variable with density

$$p(y\mid \theta,\lambda) = \prod_{i=1}^n\sum_{h=1}^H \lambda_h f(y_i\mid\theta_h).$$

Alternatively we can introduce latent class indicators $z_{ih}$

$$z_{ih} = \begin{cases}1 & \text{if i-th observation is drawn from h-th mixture component} \\ 0 &\text{otherwise}\end{cases} $$

with distribution

$$z_i = (z_{i1},\dots,z_{iH}) \mid \lambda \sim \text{Multinomial}(1; \lambda)$$

and corresponding probability mass function $p(z_i\mid\lambda)$ which lets us write

$$p(y\mid z, \theta) = \prod_{i=1}^n \prod_{h=1}^H (f(y_i\mid\theta_h))^{z_{ih}}.$$

So far things seem clear. But then the authors write:

The joint distribution of the observed data $y$ and the unobserved indicators $z$ conditional on the model parameters can be written $$p(y, z\mid\theta,\lambda) = p(z\mid\lambda)p(y\mid z, \theta) = \prod_{i=1}^n\prod_{h=1}^H(\lambda_hf(y_i\mid\theta_h))^{z_{ih}}$$ with exactly one of $z_{ih}$ equaling $1$ for each $i$.

From the perspective of probability theory we can not just multiply $p(y\mid z,\theta)$ and $p(z\mid\lambda)$, as the former is the pdf of a continous random variable and the latter is the pmf of a discrete random variable. On the other hand, we can multiply with priors for $\lambda$ and $\theta$, work out the full conditionals and get a working Gibbs sampler.

So, my question is: What is $p(y, z\mid\theta,\lambda)$ as defined above, if it is neither a pdf nor pmf?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.