Understanding conditional probability formulas in the context of class-conditionals in generative models

Question

I am trying to understand the theory behind probabilistic generative models a bit better.

If I model the class-conditionals as Gaussians, the formula is this:

$$ \frac{1}{2\pi^{\frac{D}{2}}|\Sigma|^{\frac{1}{2}}}\exp\left(-\frac{1}{2}({x}-{\mu_k)^T{\Sigma}({x}-{\mu_k})}\right) $$

My understanding is this is a conditional probability because $ \mu_k $ is used in the formula, a separate mean for each class, which is a random variable. So the outcome is now conditional on that random value.

And the $ \mu_k $ is the only notational difference between the above formula and the one for the non-conditional multivariate Gaussian for $ p(x) $.

Something similar is explained here:

https://towardsdatascience.com/3-conditionals-every-data-scientist-should-know-1916d48b078a

for the poisson, where it says that the non-conditional is $ p(y) = \frac{e^{-\lambda }\lambda ^y}{y!} $, but if $ \lambda $ itself depends on another random variable $X$, it now becomes a conditional of the form $ p(y|x) = \frac{e^{-f(X=x) }\lambda ^y}{y!} $

I understand that, when a parameter of a probability distribution depends on another random variable, we now have a conditional distribution.

However, I don't know how to relate the above two formulas for the conditional distributions to this one: $ p(x|y) = \frac{p(x,y)}{p(y)} $.

What are $p(x,y)$ and $p(y)$ in these cases?

I don't even know what theory to look at because the explanations for conditional probability are usually simple coin-toss examples. And the machine learning literature I have looked at just assumes that the reader knows this.

Simple, possibly step-by-step help is much appreciated!

Answer 1

In your multivariate normal example write the class-conditional multivariate normal density for $X$ as $f(x; \mu_k, \Sigma)$ or shorter as $f_k(x)$. Let the class $K$ be a random variable with distribution $\DeclareMathOperator{\P}{\mathbb{P}} \P(K=k)=\pi_k$. The unconditional distribution of $X$ is then a mixture distribution with the density of $X$ given by $$ f(x)=\sum_k \pi_k f(x; \mu_k, \Sigma) $$ Relating this to the formula you cite (with $y$ replaced by $k$ to conform to this example), that is, $$ p(x \mid k)= \frac{p(x,k)}{p(k)} $$ and further, for the example, replace $p$ with $f$).

Then we have: \begin{align} p(k) &= \pi_k \\ p(x,k) &= \pi_k \cdot f(x; \mu_k,\Sigma) \\ p(x\mid k)&= f(x; \mu_k,\Sigma) \end{align} Maybe one problem relating to this is that most text-book examples of jiunt distributions are either joint discrete or joint continuous, while this is an example of a mixed distribution, $X$ is continuous while $K$ is discrete.