I met a problem related to conditional probability from the article "Bayesian Networks without Tears"(download) on page 3.

According to the Figure 2, the author says $$P(fo=yes|lo=true, hb=false)=0.5$$

While I don't know how to calculate it and obtain the correct result. I attached the screenshot of Figure 2. from the article. I tried it as following $$P(fo|lo, \bar{hb})=\frac{P(lo, \bar{hb}|fo)\cdot P(fo)}{P(lo, \bar{hb})}=\frac{P(lo|fo)\cdot P(\bar{hb}|fo)\cdot P(fo)}{P(lo, \bar{hb})}$$

And $P(lo|fo)=0.6,\ P(fo)=0.15$, but how to know other 3 terms?

Thank you!

P.S. According to some reference it has $$P(\mathbf{X})=\prod_{i=1}^{n}P(X_i|parents(X_i))$$ Thus $$P(fo|lo, \bar{hb})=\frac{P(lo, \bar{hb}, fo)}{P(lo, \bar{hb})}=\frac{P(fo)\cdot P(lo|fo)\cdot P(\bar{hb}|do)}{\sum_{{fo}^{'}}P(lo, \bar{hb}, {fo}^{'})}$$ $$=\frac{.15\times.6\times.3}{.15\times.6\times.3+.85\times.05\times.3}=0.6792453$$ While it is not equal to the author's result (i.e. 0.5)? What is wrong in my solution?

enter image description here


1 Answer 1


Generally there is a very efficient algorithm called Belief Propagation, which gives exact results when the structure of the Bayesian Network is a singly connected tree (there is only a single path between any two vertices in the undirected version of the graph). You can make use of that algorithm for an exact inference in this case.

The problem is, that algorithm is somewhat complex, instead of it, you can directly use the definition of the Bayesian Network as well:

$$P(fo,bp,lo,do,hb) = P(fo)P(bp)P(lo|fo)P(do|fo,bp)P(hb|do)$$

What you want to calculate is:

$$\dfrac{P(fo=yes,lo=true,hb=false)}{P(lo=true,hb=false)}=\dfrac{\sum_{bp}\sum_{do}P(fo)P(bp)P(lo|fo)P(do|fo,bp)P(hb|do)}{\sum_{bp}\sum_{do}\sum_{fo}P(fo)P(bp)P(lo|fo)P(do|fo,bp)P(hb|do)} = \dfrac{P(lo=true,fo=yes)\sum_{bp}P(bp)\sum_{do}P(do|fo,bp)P(hb|do)} {P(lo=true,fo=yes)\sum_{bp}P(bp)\sum_{do}P(do|fo,bp)P(hb|do) +P(lo=true,fo=no)\sum_{bp}P(bp)\sum_{do}P(do|fo,bp)P(hb|do)} $$

This seems complex but since the variables are binary, the summations are just done on the two different values of the variable we are summing on. The variables behind the summation signs, which are not summed over has their values $lo=true$ and $hb=false$ but I did not write them in order to make the whole thing readable.

In general for more complex networks, this kind of variable elimination techniques are not tractable computationally but for easier graphs, it is OK.

  • $\begingroup$ Thanks! How to prove the first formula you wrote? Is it a standard theorem in Bayesian Networks? Can you kindly provide me some reference related to that formula? Cheers! $\endgroup$ Aug 20, 2014 at 7:46
  • $\begingroup$ By the way, on the most right of Figure 2. Is there a mistake of the last two equations? $\endgroup$ Aug 20, 2014 at 7:49
  • $\begingroup$ In general, the Bayesian network encodes the following relation for the joint probability of the variables it contains: $$P(x_1,x_2,...,x_N) = \prod_{i=1}^{N}P(x_i|parents_in_graph(x_i))$$ The idea is to show the independence relation between the variables using the graph theory. This is the most basic formula: You define an ordering of the variables, then you write the classic probability chain rule as:$P(x_1,x_2,...,x_N) = P(x_1)P(x_2|x_1)...P(x_N|x_{N-1},...,x_{1})$. You first write $x_1$ as a node. Then you add $x_2$ and proceed so. You add only edges to previous nodes if there is a ... $\endgroup$ Aug 20, 2014 at 7:57
  • $\begingroup$ direct dependence relation between them. If you can make a conditional independence assumption, like, "I know that the variable I am currently adding as a node is independent of the nodes in the set $A$ given the nodes in $(All current nodes)/A$. Then you add directed edges from the nodes which do not belong to $A$. This generates the first formula I have written. From there, one can mathematically produce the so called "Local Markov property" which indicates that "A node $x$ is independent of its nondescendants, given its parents". $\endgroup$ Aug 20, 2014 at 8:03
  • $\begingroup$ The existence of Local Markov Property then entails a much more general set of conditional independence relations called as the "D-separation". But this is a more advanced topic. For now, it would be sufficient to know the basic factorization definition of the Bayesian Network for the joint probability it encodes. $\endgroup$ Aug 20, 2014 at 8:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.