# What is meant by this formulation of Bayes' Rule?

From the Wikipedia article on Bayesian inference, we get the following formulation of Bayes' Rule:

$$p(\theta \mid \mathbf{X},\alpha) = \frac{p(\mathbf{X} \mid \theta) p(\theta \mid \alpha)}{p(\mathbf{X} \mid \alpha)} \propto p(\mathbf{X} \mid \theta) p(\theta \mid \alpha).$$

I do not understand this at all. How does one arrive at this equation from the humble $p(A\mid B) = \frac{p(B\mid A)p(A)}{p(B)}$? (I understand Bayes' Rule well in a Frequentist context.)

Also, what is the meaning of the numerator $p(\mathbf{X} \mid \theta) p(\theta \mid \alpha)$, and how is it different from $p(\mathbf{X} \mid \alpha)$?

• This has nothing to do with the frequentist/Bayesian quarrel. The first formula above is the probabilistic definition of the conditional density of $\theta$ given $X=x$ (I would rather write the above with $x$ instead of $X$) and the hyperparameter $a$. Jan 25, 2015 at 17:32

From basic conditional probability ideas:

$\hspace{1cm}p(\theta, X,\alpha) = p(\theta\mid X,\alpha)\, p(X,\alpha)$

also

$\hspace{1cm}p(\theta, X,\alpha) = p(X \mid \theta,\alpha)\, p(\theta,\alpha)$

Hence

$\hspace{1cm}p(\theta\mid X,\alpha) = \frac{p(X \mid \theta,\alpha)\, p(\theta,\alpha)}{p(X,\alpha)}$

$\hspace{3.3cm} = \frac{p(X \mid \theta,\alpha)\, p(\theta|\alpha)\, p(\alpha)}{p(X,\alpha)}$

$\hspace{3.3cm}\propto p(X \mid \theta,\alpha)\, p(\theta|\alpha)\, p(\alpha)$

But here $\alpha$ appears to be fixed/known, and given $\theta$, $X$ doesn't depend on $\alpha$ (i.e. $X$ is only related to $\alpha$ via $\theta$ ... $p(X|\theta,\alpha)=p(X|\theta)$), so the result then follows:

$\hspace{3.3cm}\propto p(X \mid \theta)\, p(\theta|\alpha)$

• Your final expression is $p(x\mid \theta,a)p(\theta\mid a)p(a)$. How does this relate to $p(x\mid \theta)p(\theta\mid a)$? Nov 19, 2014 at 5:28
• Okay, small edit made. Nov 19, 2014 at 5:32