# Bayes' theorem in Gaussian mixture model for $p(z_i = k | x_i, \mu_k,\Sigma_k)$

Given the $$k$$th Gaussian distribution $$N \sim (\mu_k, \Sigma_k)$$, the probability that $$x_i$$ generated from this Gaussian $$k$$ can be found via Bayes' rule \begin{align}p(z_i = k | x_i,\mu_k, \Sigma_k) &= \frac{p(x_i,z_i =k)}{p(x)} \\ &= \frac{\pi_kN(x_i|\mu_k,\Sigma_k)}{\sum_{k=1}^m\pi_kN(x_k|\mu_k,\Sigma_k)}\end{align} where $$p(x,z_i=k)$$ is the joint probability density distribution while $$p(x)$$ is the marginal distribution over the mixture of Gaussians.

Bayes' theorem in machine learning is applied in the following manner, when estimating the posterior of the model parameters $$\theta$$, $$p(\theta|D) = \frac{p(\theta)p(D|\theta)}{\int p(D|\theta)p(\theta)d\theta}$$ In this case $$p(D|\theta)$$ is a conditional probability because $$\theta$$ is a random variable.

1. why is it the case that $$N(x_i|\mu_k,\Sigma_k)$$ is not a conditional probability but can still be used in Bayes' theorem ?
2. Is the numerator in Bayes' theorem a distribution or a discrete probability? When is it the case where it is a distribution and when is it the case where the numerator is a probability. I know that $$p(\theta)p(D|\theta)$$ is a distribution over $$\theta$$ and $$\pi_kN(x_i|\mu_k,\Sigma_k)$$ is also the joint distribution.
• I corrected the first formula as it is a probability conditional on $X_i=x_i$. – Xi'an Aug 6 '20 at 14:28

## 1 Answer

1. why is it the case that $$N(x_i|\mu_k,\Sigma_k)$$ is not a conditional probability but can still be used in Bayes' theorem ?

$$N(x_i|\mu_k,\Sigma_k)$$ is a conditional probability density function. It is conditional on cluster assignment $$z_i = k$$. High-level formula is

$$p(k | X) \propto p(X|k)\, p(k)$$

1. Is the numerator in Bayes' theorem a distribution or a discrete probability? When is it the case where it is a distribution and when is it the case where the numerator is a probability. I know that $$p(\theta)p(D|\theta)$$ is a distribution over $$\theta$$ and $$\pi_kN(x_i|\mu_k,\Sigma_k)$$ is also the joint distribution.

$$p(z_i=k) = \pi_k$$ is a Bernoulli distribution (discrete), while $$p(x_i|z_i=k) = N(x_i|\mu_k,\Sigma_k)$$ is a probability density function, and the result is a probability density.

• Bayes theorem for discrete case has likelihood term $p(X|k)$ conditioned on probability whereas for the continuous variable case it is a pdf ? – calveeen Aug 6 '20 at 11:15
• – Tim Aug 6 '20 at 11:28
• I see thank you >.< – calveeen Aug 6 '20 at 11:41