# How did the posterior distribution get factorized in this manner? (bayes rule)

I am refering to this course on sampling https://www.youtube.com/watch?v=TNZk8lo4e-Q. At around minute 6, the lecturer shows on the slide the posterior probability factored as:

$$p(\Theta |X,Y)=\frac{p(Y|X,\Theta)p(\Theta))}{Z}$$ where Z is the normalizing constant.

According to the product rule, the numerator should be

$$p(\Theta )p(X|\Theta )p(Y|X,\Theta )$$

What is the reason for dropping $$p(X|\Theta )$$? Thanks alot for your help!

• Presumably the assumption is that the independent variable $X$ is independent of $\Theta$ (and may even have been decided by the experimenter) so $p(X\mid\Theta)$ is constant $p(X)$ for any particular $X$ and can be cancelled with the same constant in $Z$ Jul 26, 2021 at 15:41
• if $\Theta$ is independent of X, that means that $p(X|\Theta )$=$p(X)$. This is the density/pmf of x which is not constant for any particular X. How did u come to the conclusion that this is a constant? Jul 26, 2021 at 15:58
• with some abuse of notation $p(\Theta\mid X,Y)$ $=\dfrac{p(\Theta )p(X\mid \Theta )p(Y\mid X,\Theta )}{\int p(\Theta )p(X \mid \Theta)p(Y\mid X,\Theta )\, d\Theta}$ $=\dfrac{p(\Theta )p(X )p(Y\mid X,\Theta )}{\int p(\Theta )p(X )p(Y\mid X,\Theta )\, d\Theta}$ $= \dfrac{p(\Theta )p(Y\mid X,\Theta )}{\int p(\Theta )p(Y\mid X,\Theta )\, d\Theta}$ because the $p(X \mid \Theta)=p(X)$ do not vary with $\Theta$ Jul 26, 2021 at 16:11
• thanks! that does make sense. Since p(x) does not depend on $\theta$ it can be pulled out of the integral. But why should $\theta$ be independent of X? Aren't we learning the values of $\theta$ from X and Y, so they should be dependent of one another? Jul 26, 2021 at 16:23
• It is not my example, but the idea seems to be that $y$ (which you observe) is affected by $x$ (which you either set or observe) and $\theta$ (which is unknown but for which you have a prior distribution of belief), and you then use the likelihood to update your prior distribution to a posterior distribution for $\Theta$ Jul 26, 2021 at 16:34

Consider that you have a linear regression where $$Y$$ is your output $$X$$ is your input and $$\theta$$ is your coefficients.

Then the posterior distribution of $$\theta$$ can be expressed as

$$p(\theta|X,y)=\frac{p(\theta,X,Y)}{p(X,Y)} = \frac{p(Y|X,\theta)p(X,\theta)}{p(X,Y)}$$

however, if you see a directed acyclic graph of Bayesian linear regression you will see that $$\theta$$ affects directly $$Y$$ and is independent of $$X$$, so $$p(X,\theta)=P(X)p(\theta)$$.

$$p(\theta|X,Y)= \frac{p(Y|X,\theta)p(X)p(\theta)}{p(X,Y)}$$

and divide the denominator and numerator with $$p(X)$$ and you take

$$p(\theta|X,Y) =\frac{p(Y|X,\theta)p(\theta)}{p(Y|X)}$$

• Thanks! I have accepted this as the answer. Intuitively we can also say that theta is independent of X, because thats just saying we are applying a constant prior for every dataset that we get. Jul 26, 2021 at 17:22

In the video, the normalizing constant is given as $$Z=\int p(Y|X,\theta)p(\theta)d\theta$$

All makes sense when the original equation is

$$p(\theta|X,Y)=\frac{p(Y|X,\theta)p(\theta|X)}{Z}$$

and $$p(\theta|X)=p(\theta)$$ because it is a prior in MCMC context. This is also the case for the normalizing constant.

• Thanks, but then we still have the same question from the product rule. The numerator should be $p(Y|X,\theta)p(\theta|x)p(x)$. where did the p(x) go? Jul 26, 2021 at 16:10
• No, X is given everywhere, like $c$ here: $$p(a|b,c)=p(b|a,c)p(a|c)$$ Jul 26, 2021 at 16:32