I'm preparing slides for a lecture, and I require some guidance.

I'm only talking about discrete variables.

How would you formally define the concepts surrounding Bayesian networks?

  1. A Bayesian network is just the graph (without CPTs).
  2. A Bayesian network is a graph, together with a set of compatible CPTs.
  3. A Bayesian network is a set of CPTs (inducing the graph).

I find support for the first two options within the literature.

E.g. the books by Pearl (1986, Probabilistic reasoning...) or Koller and Friedman (2009, Probabilistic Graphical Models...) define a BN as just the graph/DAG that has to be compatible with some distribution.

Then many resources say that a BN defines a distribution (e.g. Russel and Norvig, 2009, AI - A modern Approach).

What is the more common, or more accessible way of defining BNs?

  • $\begingroup$ @william, please stop spamming the site w/ tag edits. This shouldn't be done this way. You should raise the issue on meta.CV & we can make it a synonym. $\endgroup$ – gung - Reinstate Monica May 1 '16 at 19:32
  • $\begingroup$ @Sven, please don't approve these tag edits to spam the main page w/o discussion of the issue. $\endgroup$ – gung - Reinstate Monica May 1 '16 at 19:39

If you take a Bayesian network structure and parameterize it, then it defines a probability distribution. However before adding the parameterization, it only defines constraints on the distribution. And in general, a BN may include continuous variables - so there might be no CPTs at all!

The third option - "A Bayesian network is a set of CPTs (inducing the graph)" - won't work unless you limit the possible sets of CPTs, in a way that would seem arbitrary. Not all sets of CPTs induce BN structures; there are sets of conditional independences that cannot be fully represented by a DAG. (For example, say we have variables $A, B, C$ and $D$, and only two independences: $A \perp\!\!\!\perp C | \{B,D\}$ and $B \perp\!\!\!\perp D | \{A,C\}$.)

Furthermore, even if the CPTs are compatible with a DAG, they won't necessarily identify a unique model; they will only tell you the Markov equivalence class of the generating DAG.

[Edit: Here is an example, which hopefully will clarify this point. Say we have the DAG $A \to B \to C$, and the following three CPTs:

\begin{align} \text{CPT1:} \qquad P(A=1) = .2 \end{align} \begin{align} \text{CPT2:} \qquad P(B=1|A=1) &= .7 \\ P(B=1|A=0) &= .4 \end{align} \begin{align} \text{CPT3:} \qquad P(C=1|B=1) &= .75 \\ P(C=1|B=0) &= .5 \end{align}

CPT1 and CPT2 can be combined to produce the joint pmf $P(A,B)$, like so: \begin{align} P(A,B): \qquad P(A=1\; \& \;B=1) &= .14 \\ P(A=0\; \& \;B=1) &= .32 \\ P(A=1\; \& \;B=0) &= .06 \\ P(A=0\; \& \;B=0) &= .48 \\ \end{align}

The joint pmf can then be factored using the chain rule into CPT1' and CPT2':

\begin{align} \text{CPT1':} \qquad P(B=1) = .46 \end{align} \begin{align} \text{CPT2':} \qquad P(A=1|B=1) &= \frac{.14}{.46} \\ P(A=1|B=0) &= \frac{.06}{.54} \end{align}

Clearly $P(A,B,C)$ can be represented by either CPT1, CPT2 and CPT3, or alternatively by CPT1', CPT2' and CPT3. This illustrates that the distribution is compatible with two distinct DAGs: $A \to B \to C$, and $A \leftarrow B \to C$. This is because these two DAGs entail the same conditional independences (i.e. they are members of the same Markov equivalence class).]

So I would stick with the first option. In more detail:

A Bayesian network induces conditional independence constraints on all probability distributions over the nodes of the graph. In particular, a graph $G$ over a set of variables $\mathbf{X}$ entails that the probability distribution $P(\mathbf{X})$ should factor into $\Pi_i P(X_i|\mathbf{Pa}_i)$, where $\mathbf{Pa}_i$ is the set of parents of node $X_i$ in $G$. If all the variables in $\mathbf{X}$ are discrete, this means $P(\mathbf{X})$ can be characterized by a set of small CPTs. A fully parameterized discrete BN is a graph together with a set of CPTs, one for each factor in the distribution.

(If you are looking for good teaching resources I recommend David Heckerman's classic A Tutorial on Learning with Bayesian Networks (pdf). There is an associated set of slides that might save some time.)

| cite | improve this answer | |
  • $\begingroup$ Thanks for your answer. Can you please clarify why a set of CPTs compatible with a DAG don't identify a unique model. I'm only using discrete variables, btw. $\endgroup$ – ziggystar Nov 6 '14 at 23:05
  • $\begingroup$ You're welcome! I've added an example to illustrate the point, I hope that helps. $\endgroup$ – Lizzie Silver Nov 6 '14 at 23:49
  • $\begingroup$ CPT 1,2,3 do uniquely define a pmf. Maybe I missinterpret what you mean by "identify". In your example, C->B->A is ruled out by the first set of CPTs (though not by the independency-structure they induce). What I think: A set of CPTs uniquely defines a DAG (by telling the set of parents for all nodes). So it's pointless to provide a set of CPTs and a DAG, because it has to be the one DAG that is uniquely determined by the CPTs. $\endgroup$ – ziggystar Nov 7 '14 at 8:29
  • $\begingroup$ @ziggystar Ah, I think we agree. You're saying that the choice of the conditioning sets in the CPTs tells you what DAG the author intended to refer to. One may of course describe the same pmf using different conditioning sets. I should have said that the pmf does not identify the DAG (except in special cases). Yes, you could use the choice of conditioning sets in the CPTs to implicitly communicate the DAG structure. I think that is not always standard practice, and mentioning the DAG explicitly may help clarify. $\endgroup$ – Lizzie Silver Nov 7 '14 at 15:58
  • $\begingroup$ Yes, it's not standard practice; that's my dilemma. Pearl's definition of a BN is fine, as he calls only the graph a BN. But many people use the term BN to also refer to a distribution. And this might be the reason that many people define a BN as a graph plus CPTs (as Heckermann does). But in my opinion that's too redundant. $\endgroup$ – ziggystar Nov 7 '14 at 16:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.