In treatments of Bayesian methods you typically see terms combining joint distributions with conditional distributions such as $P(A,B|C)$ and $P(A|B,C)$. Expressions arise such as the chain rule $$ P(A,C|B) = P(A|B,C) P(C|B). $$

I find I'm often caught out by such manipulations. For example, moving the $C$ from the LHS of the conditional sign to the RHS we need to multiply by $P(C|B)$, not by $P(C)$.

To tune my intuition, what I would like to see is an algebra of the joint and conditional 'operators' (comma and mid) but try as I might I haven't been able to find such an algebra or derive it myself from first principles. Does such an algebra exist?


1 Answer 1


All the algebra you need is the chain rule.

$$ P(X, Y) = P(X | Y) P(Y) = P(Y | X) P(X) $$ Here $X$ and $Y$ may mean random events that are composed of other random events. The same is true for densities. Also, Bayes theorem is a corollary of this formula:

$$ P(X|Y) = \frac{P(Y|X) P(X)}{P(Y)} = \frac{P(X, Y)}{P(Y)} $$

From the chain rule you can derive $$ P(A, C | B) = \frac{P(A, \overbrace{C, B}^{Y})}{P(B)} = \frac{P(A | B,C) P(B, C)}{P(B)} = \frac{P(A | B, C) P(C|B) P(B)}{P(B)} = P(A|B,C) P(C|B) $$

You can treat this as the Conditioning Operator Rule: if you have 2 sets of random events $A$ and $B$ and you want to "split" them, you should do it accounting for possible dependence: one term's condition box should contain another (But not both). Also, you keep your conditions: in general case you can't drop it (unless you have independence). $$ P(A, B | C) = P(A | B, C) P(B | C) $$ (this can be viewed as a generalization of the first formula. Indeed, if you set $C = \Omega$ (the set of all outcomes), you can derive it)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.