Skip to main content
4 of 5
edited tags
kjetil b halvorsen
  • 82.8k
  • 32
  • 201
  • 663

Understanding conditional probability formulas in the context of class-conditionals in generative models

I am trying to understand the theory behind probabilistic generative models a bit better.

If I model the class-conditionals as Gaussians, the formula is this:

$$ \frac{1}{2\pi^{\frac{D}{2}}|\Sigma|^{\frac{1}{2}}}\exp\left(-\frac{1}{2}({x}-{\mu_k)^T{\Sigma}({x}-{\mu_k})}\right) $$

My understanding is this is a conditional probability because $ \mu_k $ is used in the formula, a separate mean for each class, which is a random variable. So the outcome is now conditional on that random value.

And the $ \mu_k $ is the only notational difference between the above formula and the one for the non-conditional multivariate Gaussian for $ p(x) $.

Something similar is explained here:

https://towardsdatascience.com/3-conditionals-every-data-scientist-should-know-1916d48b078a

for the poisson, where it says that the non-conditional is $ p(y) = \frac{e^{-\lambda }\lambda ^y}{y!} $, but if $ \lambda $ itself depends on another random variable $X$, it now becomes a conditional of the form $ p(y|x) = \frac{e^{-f(X=x) }\lambda ^y}{y!} $

I understand that, when a parameter of a probability distribution depends on another random variable, we now have a conditional distribution.

However, I don't know how to relate the above two formulas for the conditional distributions to this one: $ p(x|y) = \frac{p(x,y)}{p(y)} $.

What are $p(x,y)$ and $p(y)$ in these cases?

I don't even know what theory to look at because the explanations for conditional probability are usually simple coin-toss examples. And the machine learning literature I have looked at just assumes that the reader knows this.

Simple, possibly step-by-step help is much appreciated!