I read two different papers on some similar problems. In one of the papers this statement is written:

$ p(a|b) = \sum_{c \in C}p(a,c|b) $

While in the other it is written as:

$ p(a|b) = \sum_{c \in C}p(a|c)p(c|b) $

Therefore, this expression should hold:

$ p(a,c|b) = p(a|c)p(c|b) $

I tried to prove it but failed at some point. Are they the same? Or am I missing something? Or maybe each of them are correct under a specific assumption?

  • 5
    $\begingroup$ second line should read $p(a,c|b) = \sum p(a|c,b)p(c|b)$ right? $\endgroup$ – bdeonovic Apr 27 '16 at 11:14
  • $\begingroup$ @bdeonovic Yes it should be. $\endgroup$ – Moh Apr 27 '16 at 13:53

At the second article they should actually write: $$ p(a|b) = \sum_{c\in C} p(a|c,b) p(c|b) $$ (unless $p(a|c)=p(a|c,b)$ for all $b$).

The equation $p(a,c|b)=p(a|c,b)p(c|b)$ is correct.

By definition, $$p(x|y)=N(x,y)/N(y) = p(x, y) / p(y)$$ where $N(x,y)$ is the number of cases when both $x$ and $y$ hold, while $N(y)$ is the number of cases when $y$ holds.

Therefore, $$ p(a,c|b) = N(a,c,b)/N(b) $$ $$ p(a|c,b) = N(a,c,b)/N(c,b) $$ $$ p(c|b) = N(c,b)/N(b) $$

We can easily see that the equation is correct.

  • $\begingroup$ I read somewhere this statement: $p(a|c,b)=p(a|c)$ under the assumption that $a$ is conditionally independent of $b$ given $c$. Does it make any sense to you in this example? It might have been their assumption. $\endgroup$ – Moh Apr 27 '16 at 13:51
  • $\begingroup$ By definition, $p(a|b) = p(a,b)/p(b)$. If a and b independent, $p(a,b) = p(a)p(b)$, hence $p(a|b) =p(a)$. Condition on $c$, you get what you want... $\endgroup$ – dv_bn Apr 27 '16 at 13:52
  • $\begingroup$ "I read somewhere this statement: p(a|c,b)=p(a|c) under the assumption that a is conditionally independent of b given c." Yes, you are right, the statement is true. $\endgroup$ – user31264 Apr 27 '16 at 13:54
  • $\begingroup$ In the second article it is written that we write this marginalization based on the independence assumption of $c$ and $b$. In that case $p(c|b)$ will be $p(c)$ and makes the whole statement wrong. Am I right? $\endgroup$ – Moh Apr 27 '16 at 13:55
  • $\begingroup$ @dv_bn Could you please give me an example? $\endgroup$ – Moh Apr 27 '16 at 13:57

Start from the identity $P(a,c) = P(a|c)P(c)$.

Now condition both sides with respect to $b$ and obtain

$P(a,c|b) = P(a|c,b)P(c|b)$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.