So I understand that using sum-rule one can write a probability as a marginal:

\begin{equation*} P(x) = \int{P(x,\theta)d\theta} = \int{P(x|\theta)P(\theta)}d\theta \end{equation*}

But how is this applied to a conditional probability? For instance in bayesian model comparison I come across this formula a lot:

\begin{equation*} P(\mathcal{D}|m) = \int{P(\mathcal{D}|\theta,m)}P(\theta|m)d\theta \end{equation*}

First of all I'm confused about the notation. Where are the parentheses in $P(\mathcal{D}|\theta,m)$? Is it $P(\mathcal{D}|(\theta,m))$ or is it $P((\mathcal{D}|\theta),m)$? And how is it derived anyway?

  • $\begingroup$ it is exactly the same, $m$ being fixed all over. No need for further parentheses, with $P((\mathcal{D}|\theta),m)$ being not a meaningful notion. $\endgroup$
    – Xi'an
    Aug 9 '18 at 21:43

Its $P(\mathcal{D}|\theta,m)$ and it denotes the conditional of $\mathcal{D}$ given both $\theta$ and $m$.

\begin{equation*} P(\mathcal{D}|m) = \int{P(\mathcal{D},\theta|m)}d\theta = \int{P(\mathcal{D},\theta,m)/P(m)}d\theta = \int{P(\mathcal{D}|\theta,m)P(\theta,m)/P(m)}d\theta = \int{P(\mathcal{D}|\theta,m)}P(\theta|m)d\theta, \end{equation*} where the first equality follows in the same way as the first equation you wrote.

The intuition is exactly the same as the first equation. But since $m$ was given to begin with, it will appear in all conditionals in the right-hand side.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.