Clarifying Dirichlet Process Mixture Probability Terms Suppose I have a Dirichlet Process Mixture model defined as follows:
$\alpha \sim G(a,b)\\
\pi|\alpha \sim \text{Dir}(\alpha)\\
z|\pi \sim \text{Cat}(\pi)\\
$
where $G$ is just a standard Gamma distribution, and 
$
\mu,\Sigma \sim NIW(\beta)
$
where $NIW$ stand for Normal Inverse Wishart.
Ultimately the plate model looks like:

Upon doing some probabilistic maniuplation I arrive at the following terms I would like some clarification on: $p(z|X),p(z|X,\alpha),p(X|z)$. I am just wondering how to interpret these.


*

*Is $p(z|X,\alpha) = p(z|\alpha)$? It seems that $X$ is already implicitly used in the $p(z|\alpha)$ expression as the `number of terms allocated to a cluster' in the sense of the Chinese Restaurant Process, when considering $p(z|\alpha)$ by itself, so does conditioning on $X$ change anything?

*How should I interpret the expressions $p(z|X)$ and $p(X|z)$? It feels strange because $z$ just acts like a `switch' or index for this Bayes Net, so interpreting either direction of these feels strange. 

*In addition, if I add a (Gamma) prior distribution over the $\alpha$ term, with a likelihood term similar to that suggested by Escobar & West 1995 (equations 13 / 14) then is it possible to make the claim that this empirical distribution over $\alpha$ has a regularising effect over the amount of clusters forming (or in the cluster allocations for each point, that is $p(z_i = k | X, \beta, \alpha, z_{\lnot i}$).
 A: Let's review the generative process assumed to generate data using GMM with infinite (i.e. non-fixed) number of clusters.


*

*First we need to choose a cluster assignment. Using a Chinese restaurant process, we assume that there are currently $K$ clusters, but we also have a probability to assign an observation to a new cluster $K+1$. This way we do not need to fix $K$ a priori, 


$$
\begin{equation}\tag{1}\label{eqn1}
  P(z_i=k\mid \alpha) =
    \begin{cases}
      \frac{N_k}{N+\alpha-1} & , k \in [1, K]\\
      \frac{\alpha}{N+\alpha-1} & , k = k_{new}=K+1\\
    \end{cases}       
\end{equation}
$$


*Given a cluster assignment, we can generate an observation from the corresponding Gaussian with parameters: $\mathcal{N}(\mu_k, \Sigma_k)$
For clustering, we need to determine the probability of assigning an observation to a cluster, which can be done using Bayes rule as follows,
$$\tag{2}\label{eqn2}
P(\textbf{z} \mid \textbf{X}, \alpha, \beta) \propto P(\textbf{X} \mid \textbf{z}, \beta) \times P(\textbf{z} \mid \alpha)
$$
If we are doing MCMC sampling, the above equation can be written as,
$$\tag{3}\label{eqn3}
P(z_i=k \mid \textbf{z}_{-i}, \textbf{X}, \alpha, \beta) \propto P(x_i \mid \textbf{X}_{-i}, z_i=k, \textbf{z}_{-i}, \beta) \times P(z_i=k \mid \textbf{z}_{-i}, \alpha)
$$

Answering your questions:

Is $P(\textbf{z} \mid \textbf{X}, \alpha, \beta) = P(\textbf{z} \mid \alpha)$?

No they are not. $P(\textbf{z} \mid \textbf{X}, \alpha, \beta)$ is the posterior evaluated using equation $(\ref{eqn2})$ and you get $P(\textbf{z} \mid \alpha)$ by integrating out $\pi$ as follows:
$$
P(\textbf{z} \mid \alpha) = \int_{\pi}P(\textbf{z} \mid \mathbb{\pi}) \: p(\mathbb{\pi} \mid \alpha) \: d\pi
$$

How should I interpret the expressions $P(\textbf{z} \mid \textbf{X})$ and $P(\textbf{X} \mid \textbf{z})$?

The former term is the posterior distribution $[$ i.e. which is (\ref{eqn2}) ignoring $\alpha$ and $\beta$ $]$ of assigning clusters to the data, the latter is the likelihood of the data. The posterior is calculated using the two terms: 


*

*$P(z_i=k \mid \alpha)$ is given by equation $(\ref{eqn1})$ 

*$P(x_i \mid z_i=k, \beta) \sim \mathcal{N}(\mu_k, \Sigma_k)$

... then is it possible to make the claim that this empirical distribution over $\alpha$ has a regularising effect over the amount of clusters forming ...

Your analysis is correct, since the choice of $\alpha$ governs how many clusters will be allocated and whether the model will have a tendency to favor existing clusters or generate more clusters. 

References:


*

*Gibbs sampling for fitting finite and infinite Gaussian mixture models - by Herman Kamper

*A tutorial on Bayesian nonparametric models - by Gershman, Blei
