I really don't understand what is the purpose of the Bayes network, actually how to implement it into a useful application.

It all starts with the data. Let's assume I observe some universe and I have three random variables $A,B,C$. From there I can create a joint distribution table (see table below). As far as I understand it, this is my solid base, from there on I can use some probability laws and infer everything that I would like and I am done.

enter image description here

And now let's go to a problem. If I understand correctly Bayes networks are based on assumption. Therefore one can say (according to a data above), we can make the following model:

enter image description here

$P(A,B,C) = P(C|A,B) \cdot P(B|A) \cdot P(A)$

following conditional independence:

$P(A,B,C) = P(C|A,B) \cdot P(B) \cdot P(A)$

if we would infer $P(B|C,A) = P(C|A,B) \cdot P(B) / P(C|A)$

But just look the joint probability table above. Plug the data to formula and you would see how wrong this assumption is.

Just looking at the table: $P(B=1|C=1,A=1) = 0.31$

Using Bayes network:

$P(B=1| C= 1, A=1) = P(C=1|A=1,B=1) \cdot P(B=1) / P(C=1|A=1)$ results in 0.42

So my point is:

  1. If you use joint probability table only, you get result from data, which is accurate and you don't have to assume anything. Computing gets more difficult if you have more random variables, but o.k...

  2. To use Bayes network and create a graph, you need to make a very expensive assumption of your model. As more random variables you have, it's more likely you fail with assumption. I have presented you assumption on conditional independence can give you totally wrong result and it's a simple 3 random variable model. Now think if size would increase.

But, I know I fail to see something very important and I would appreciate if you could help me understand this better. Thank you.

  • $\begingroup$ For clarification: how did you get $P(B=1 | C=1, A=1) = 0.31$? Because there are two rows where both $A$ and $C$ equal 1, and for these $B=1$ in one row. Am I missing something here? $\endgroup$ May 30, 2020 at 19:44
  • $\begingroup$ Yes, two rows where A and C are 1. I summed this two rows, which is 0,0889... and then I divided first row with 0,0889, and you get 0,31 approx. $\endgroup$
    – Stenga
    May 30, 2020 at 19:57
  • $\begingroup$ Ah, I overlooked the last columns. So it is basically just 18/(18+40) in that case. Got it. $\endgroup$ May 30, 2020 at 20:02
  • $\begingroup$ @horseoftheyear yes, that would be right. and thank you for edit:) $\endgroup$
    – Stenga
    May 30, 2020 at 20:03

1 Answer 1


Bayes Nets saves you from needing to estimate and store a large number of parameters. If you have $V$ variables with $D$ possible values each, to create the joint distribution table you would need to estimate and store $O(D^V)$ different parameters. In real problems with 100s of variables with many possible values each, this is clearly impractical. You'll never have enough data to create the table (and likely run out of computational resources as well).

By using the conditional Independence relations you can get away with estimating a much smaller number of parameters. But of course, as your example shows, you need to ensure that you have the conditional independence structure right - otherwise you will not get correct answers.

But at the same time in a real problem, you have limited amount of data, and your computing resources are not unbounded. So even if you get the structure wrong, Bayes net type approach may still be the only viable option. And of course, there are also methods to actually estimate the conditional independence structure - but learning the structure is generally considered difficult.

Edited to add: the wikipedia article seems quite good.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.