1
$\begingroup$

Learning about Bayesian Networks in school - I ran across a problem which ask to find the probability of $Pr(Alarm|Storm=T)$ given a column of event data for four variables: Storm, Burglar, Cats, and Alarm where all these variables are either true or false.

The Bayes Network solution apparently is the following:

enter image description here

  • I did not fully understand why we are multiplying the intermediate conditionals i.e.: $... Pr(Burglar|Storm=T) x Pr(Cat|Storm=T) ...$

I thought a more bayesian way to solve this is with probability trees:

enter image description here

Perhaps my other way it is not right but the answer turns out the same.

So, my question is where is the bayes in bayes networks? What is the relationship between bayes rule (not naive bayes) and bayesian networks?

Is there a logical explanation for the multiplication of the intermediate probabilities - or is it just a trick?

$\endgroup$
2
$\begingroup$

From Wikipedia:

A Bayesian network (also known as a Bayes network, belief network, or decision network) is a probabilistic graphical model that represents a set of variables and their conditional dependencies via a directed acyclic graph (DAG).

Bayes' rule is used for inference in Bayesian networks, as will be shown below. A better name for a Bayesian network would be directed probabilistic graphical model, and the main purpose of a probabilistic graphical model is to efficiently represent the conditional independencies in a joint probability distribution. The fundamental idea to understanding the conditional independencies represented by Bayesian networks is the Markov assumption:

The Markov condition, sometimes called the Markov assumption, is an assumption made in Bayesian probability theory, that every node in a Bayesian network is conditionally independent of its nondescendents, given its parents. Stated loosely, it is assumed that a node has no bearing on nodes which do not descend from it. In a DAG, this local Markov condition is equivalent to the global Markov condition, which states that d-separations in the graph also correspond to conditional independence relations. This also means that a node is conditionally independent of the entire network, given its Markov blanket.

So, in your diagram, using the Markov assumption, the following conditional independencies can be deduced:

  1. $\text{Storm} \perp \emptyset \ | \ \emptyset$
  2. $\text{Burglar} \perp \text{Cat} \ | \ \text{Storm}$
  3. $\text{Cat} \perp \text{Burglar} \ | \ \text{Storm}$
  4. $\text{Alarm} \perp \text{Storm} \ | \ \{\text{Cat},\text{Burglar}\}$

Where $\text{A} \perp \text{B} \ | \ \text{C}$ reads as "$\text{A}$ is conditionally independent of $\text{B}$ given $\text{C}$", and $\emptyset$ is the empty set. The joint probability distribution is: $$ p(\text{Storm},\text{Burglar},\text{Cat},\text{Alarm}) = p(\text{Alarm}|\text{Storm},\text{Burglar},\text{Cat}) \cdot p(\text{Burglar}|\text{Storm},\text{Cat}) \cdot p(\text{Cat}|\text{Storm}) \cdot p(\text{Storm}) $$ Given the conditional independencies that were deduced above, then this joint distribution can be simplified to: $$ p(\text{Storm},\text{Burglar},\text{Cat},\text{Alarm}) = p(\text{Alarm}|\text{Burglar},\text{Cat}) \cdot p(\text{Burglar}|\text{Storm}) \cdot p(\text{Cat}|\text{Storm}) \cdot p(\text{Storm}) $$ Next, using Bayes' rule, we want to find: $$ p(\text{Alarm}|\text{Storm}) = \frac{p(\text{Alarm},\text{Storm})}{p(\text{Storm})} $$ Using the law of total probability, we know that: $$ \begin{align} p(\text{Alarm},\text{Storm}) &= \sum_{\text{Burglar}} \sum_{\text{Cat}} p(\text{Storm},\text{Burglar},\text{Cat},\text{Alarm}) \\ &= \sum_{\text{Burglar}} \sum_{\text{Cat}} p(\text{Alarm}|\text{Burglar},\text{Cat}) \cdot p(\text{Burglar}|\text{Storm}) \cdot p(\text{Cat}|\text{Storm}) \cdot p(\text{Storm}) \\ &= p(\text{Storm}) \sum_{\text{Burglar}} p(\text{Burglar}|\text{Storm}) \sum_{\text{Cat}} p(\text{Alarm}|\text{Burglar},\text{Cat}) \cdot p(\text{Cat}|\text{Storm}) \end{align} $$ So: $$ p(\text{Alarm}|\text{Storm}) = \sum_{\text{Burglar}} p(\text{Burglar}|\text{Storm}) \sum_{\text{Cat}} p(\text{Alarm}|\text{Burglar},\text{Cat}) \cdot p(\text{Cat}|\text{Storm}) $$ An efficient way of computing $p(\text{Alarm}|\text{Storm})$ as it is shown above is belief propagation.

$\endgroup$
6
  • $\begingroup$ I don't agree that they have "no relation to Bayes' rule"; Bayes's rule is pretty crucial to inference. In all, it's a good answer. $\endgroup$ – Arya McCarthy Apr 12 at 14:18
  • $\begingroup$ @AryaMcCarthy edited to clarify. Thanks for commenting. $\endgroup$ – mhdadk Apr 12 at 14:34
  • $\begingroup$ Ahh so $p(Burglar|Storm,Cat)$ equal $p(Burglar|Storm)$ because cats and burglars are independent. I think this explains the derivation - I still dont understand why we have to multiply $p(Burglar|Storm)$ and $p(Cat|Storm)$ $\endgroup$ – Edv Beq Apr 12 at 16:20
  • $\begingroup$ Actually i get it now - so, because they are independent the joint is $p(Burglar, Cat, Storm) = p(Burglar|Storm)p(Cat|Storm)$ $\endgroup$ – Edv Beq Apr 12 at 16:28
  • 1
    $\begingroup$ Hi @EdvBeq, your probability trees approach is fine. However, because it does not make use of the conditional independencies given by the Bayesian network, it is a bit inefficient and will not scale well for a large number of random variables. Have a look at belief propagation too, as it is very efficient. $\endgroup$ – mhdadk Apr 12 at 17:41
0
$\begingroup$

In response to

I did not fully understand why we are multiplying the intermediate conditionals i.e.: $...Pr(Burglar|Storm=T)xPr(Cat|Storm=T)...$

and

Is there a logical explanation for the multiplication of the intermediate probabilities - or is it just a trick?

Denoting each random variable with their first letter, the model represented using the Bayesian network suggests the following factorisation of the joint probability distribution:

$$P(S, B, C, A) = P(S)P(B|S)P(C|S)P(A|B,C)$$

Via marginalisation, we have that

$$\begin{align}P(A, S) &= \sum_{b \in \{0, 1\}} \sum_{c \in \{0, 1\}} P(S, B =b, C=c, A) \\ &= \sum_{b \in \{0, 1\}} \sum_{c \in \{0, 1\}} P(S)P(B = b | S) P(C=c|S)P(A|B=b,C=c) \end{align}$$

Which can be used to compute $P(A | S)$ via Bayes theorem

$$P(A | S) = \frac{P(A, S)}{P(S)}$$

Lastly, in response to

So, my question is where is the bayes in bayes networks? What is the relationship between bayes rule (not naive bayes) and bayesian networks?

Bayesian networks are also referred to as directed acyclic graphs. Whilst you do often use Bayes theorem repeatedly when using these representations, and the representation is often associated with a Bayesian statistical approach, most texts I have seen on the subject tend to emphasise that one should not read too much into the naming. And to the best of my knowledge the name is due to historical associations rather than any deliberate semantic choice.

$\endgroup$
2
  • $\begingroup$ Personally, I also find the example somewhat confusing, in that there are two distinct notations being used for denoting the state of the random variable. On the one hand, you have $Cat$ and $notCat$, suggesting a parallel with event notation of $C$ and its complement $C'$. And then at the same time you have $Storm = T$, where I assume $T$ stands for true. In all cases, the confusion could be eliminated by using consistent notation indicating the realisation of the random variable $C = 0$ and $C = 1$ for all random variables. $\endgroup$ – microhaus Apr 12 at 13:50
  • $\begingroup$ Point well taken on the notation $\endgroup$ – Edv Beq Apr 12 at 13:58
0
$\begingroup$

In Bayesian Networks, there are three types of junctions, https://en.wikipedia.org/wiki/Bayesian_network. In your case, you have a Fork junction, $Storm \rightarrow Burglar \ \& \ Storm \rightarrow Cat.$ Each junction case has some conditional or unconditional independence properties. For your case, the Fork junction has

$$Burglar \perp Cat \ | \ Storm $$

which gives you the conditional independence of the densities.

And I think the Bayes rule comes in the following way:

$$p(A|S=T) = \frac{p(A,S=T)}{p(S=T)}=\frac{\sum_{B}\sum_{C}p(A,B,C,S=T)}{p(S=T)}$$ where $p(A,B,C,S=T)$ can be read out from the directed graph and by using the Fork junction independences, $p(A,B,C,S=T)=p(S=T)p(C|S=T)p(B|S=T)p(A|B,C).$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.