# Explicitly Show Conditional Independence In a Mixture Model

Suppose we have the following mixture model:

$$\pi \sim Dirichlet(\alpha)$$

$$\theta_1 ,\ldots ,\theta_K \overset{iid}{\sim} N(0,1)$$

$$Z_1 , \ldots , Z_n \mid \pi \overset{iid}{\sim} Categorical(\pi)$$

$$y_1 , \ldots , y_n \mid Z_1, \ldots, Z_n, \theta_1 , \ldots ,\theta_K \overset{indep}{\sim} N(\theta_{Z_i},1)$$.

When we write the joint distribution, we say

$$p(\pi,\theta,Z,Y) = p(\pi)p(\theta)p(Z\mid \pi)p(y\mid Z,\theta)$$

where $\theta=(\theta_1 ,\ldots, \theta_K)$ and $Z=(Z_1,\ldots,Z_n) , Y = (y_1 ,\ldots y_n)$.

This assumes that $Y$ is conditionally independent of $\pi$, given $Z, \theta$. I think this seems intuitively obvious since the conditional distribution of $Y\mid Z,\theta$ is free of $\pi$. But how do we explicitly prove this given the model setup?

Or, is the model just codifying an assumption, in which case there is nothing to prove?

I came across the same question by myself when I'm studying a Bayesian book. And I guess I have possible answers (one is a detailed explanation and another is a simple one that appeals your experience in multivariate calculus class).

I. One Possible Answer (detailed explanation)

When you specify the model as you did above by $$p(Y|Z, \theta)$$, not by $$p(Y|Z, \theta, \pi)$$, then it's like you are incorporating the "information" or "knowledge" (such that $$Y$$ is not directly dependent on $$\pi$$) about your model and restricting the form of the joint distribution into simpler form, as Tom Loredo explains the notion of conditional independence in https://math.stackexchange.com/questions/23093/could-someone-explain-conditional-independence. Or, you can think $$y_1, ..., y_n | Z_1, ..., Z_n, \theta_1, ..., \theta_K \sim p(Y|Z, \theta, \pi) := \mathcal{N}(\theta_{z_i}, 1) = p(Y|Z, \theta)$$.

If you add another model specification like $$p(Y|\pi)$$ or $$p(Y|Z, \theta, \pi)$$ that is not equal to $$p(Y|Z, \theta)$$ to the model you specified above, I think such a model still makes sense while you will lose conditional independence due to the Bayes rule (think of the example of a discrete case below). Hence there is no proof about the conditional independence but the conditional independence is rather the "specification" of the model you work with.

EXAMPLE OF DISCRETE CASE: Think of defining $$P(W|R, C)$$ that is not $$0.9$$ in the video below of discrete case example will still makes sense but changes the structure of graphical model and lose conditional independence (let's say $$P(W|R, C) = 0.95, P(W|R, \sim C) = 0.85, P(W|\sim R, C) = 0.3, P(W|\sim R, \sim C) = 0.05$$. In this case $$P(W|R)$$ is calculated as $$0.9342105$$ hence $$P(W|R, C) \neq P(W|R)$$, which means $$C$$ and $$R$$ are not conditionally independent given $$W$$. Note we can even keep $$P(W|R) = 0.9$$ and omit some of $$P(W|R, C)$$, $$P(W|\sim R, C)$$, $$P(W|R, \sim C)$$ or $$P(W|\sim R, \sim C)$$). https://www.youtube.com/watch?v=WVKFaDqcBFQ

Consider a simple multivariate function $$f(x, y)$$. Given this specification of the function $$f$$, you would assume $$x$$ and $$y$$ are independent unless $$y$$ is specified as $$y = g(x)$$, or $$y(x)$$. I suppose the same kind of situation is happening in graphical model specifications.