Explicitly Show Conditional Independence In a Mixture Model

Question

Suppose we have the following mixture model:

$$\pi \sim Dirichlet(\alpha)$$

$$\theta_1 ,\ldots ,\theta_K \overset{iid}{\sim} N(0,1)$$

$$Z_1 , \ldots , Z_n \mid \pi \overset{iid}{\sim} Categorical(\pi)$$

$$y_1 , \ldots , y_n \mid Z_1, \ldots, Z_n, \theta_1 , \ldots ,\theta_K \overset{indep}{\sim} N(\theta_{Z_i},1)$$.

When we write the joint distribution, we say

$$p(\pi,\theta,Z,Y) = p(\pi)p(\theta)p(Z\mid \pi)p(y\mid Z,\theta)$$

where $\theta=(\theta_1 ,\ldots, \theta_K)$ and $Z=(Z_1,\ldots,Z_n) , Y = (y_1 ,\ldots y_n)$.

This assumes that $Y$ is conditionally independent of $\pi$, given $Z, \theta$. I think this seems intuitively obvious since the conditional distribution of $Y\mid Z,\theta$ is free of $\pi$. But how do we explicitly prove this given the model setup?

Or, is the model just codifying an assumption, in which case there is nothing to prove?

Answer 1

I came across the same question by myself when I'm studying a Bayesian book. And I guess I have possible answers (one is a detailed explanation and another is a simple one that appeals your experience in multivariate calculus class).

I. One Possible Answer (detailed explanation)

When you specify the model as you did above by $p(Y|Z, \theta)$, not by $p(Y|Z, \theta, \pi)$, then it's like you are incorporating the "information" or "knowledge" (such that $Y$ is not directly dependent on $\pi$) about your model and restricting the form of the joint distribution into simpler form, as Tom Loredo explains the notion of conditional independence in https://math.stackexchange.com/questions/23093/could-someone-explain-conditional-independence. Or, you can think $y_1, ..., y_n | Z_1, ..., Z_n, \theta_1, ..., \theta_K \sim p(Y|Z, \theta, \pi) := \mathcal{N}(\theta_{z_i}, 1) = p(Y|Z, \theta)$.

If you add another model specification like $p(Y|\pi)$ or $p(Y|Z, \theta, \pi)$ that is not equal to $p(Y|Z, \theta)$ to the model you specified above, I think such a model still makes sense while you will lose conditional independence due to the Bayes rule (think of the example of a discrete case below). Hence there is no proof about the conditional independence but the conditional independence is rather the "specification" of the model you work with.

EXAMPLE OF DISCRETE CASE: Think of defining $P(W|R, C)$ that is not $0.9$ in the video below of discrete case example will still makes sense but changes the structure of graphical model and lose conditional independence (let's say $P(W|R, C) = 0.95, P(W|R, \sim C) = 0.85, P(W|\sim R, C) = 0.3, P(W|\sim R, \sim C) = 0.05$. In this case $P(W|R)$ is calculated as $0.9342105$ hence $P(W|R, C) \neq P(W|R)$, which means $C$ and $R$ are not conditionally independent given $W$. Note we can even keep $P(W|R) = 0.9$ and omit some of $P(W|R, C)$, $P(W|\sim R, C)$, $P(W|R, \sim C)$ or $P(W|\sim R, \sim C)$). https://www.youtube.com/watch?v=WVKFaDqcBFQ

II. Another Possible Answer (simple explanation that appeals your experience)

Consider a simple multivariate function $f(x, y)$. Given this specification of the function $f$, you would assume $x$ and $y$ are independent unless $y$ is specified as $y = g(x)$, or $y(x)$. I suppose the same kind of situation is happening in graphical model specifications.

Hope these might help someone or activate the discussion, and sorry in advance if there is some error or unclarity in my answer since this is my first post to stack exchange.