Finding the corresponding bayesian network of a predefined joint probability distribution

Question

Given a joint probability distribution over the variables $X_1,X_2,\dots,X_n$. Is there an algorithm for constructing the corresponding Bayesian Network?

Answer 1

First a quick note, since your question asks about "constructing the corresponding Bayesian network," singular rather than plural. In general, the joint distribution of $X_1,\dots,X_n$ doesn't determine a unique such structure, which makes sense, as the Bayesian network is intended to model causality, and a joint distribution in and of itself has nothing to say about that matter.

The directed acyclic graph involved is just a graphical representation of one particular factorization of the joint distribution. Consider, say, two dependent variables $Y_1$ and $Y_2$. One factorization of the joint distribution is indicated by

$$\mathbb{P}(Y_1=y_1,Y_2=y_2) = \mathbb{P}(Y_1=y_1)\mathbb{P}(Y_2=y_2 \mid Y_1=y_1),$$

which would correspond to the DAG with an edge $Y_1 \longrightarrow Y_2$, but it could just as well be factorized with the roles of $Y_1$ and $Y_2$ flipped, yielding the DAG with an edge $Y_2 \longrightarrow Y_1$. Their joint distribution is simply agnostic about the causal story.

All that said, there are indeed a variety of algorithms out there for (1) determining causal structures that are consistent (by some measure) with a given joint distribution, and (2) ranking or scoring such structures in terms of some fit. For instance, the SGS algorithm of Spirtes, Glymour and Scheines is a well-known algorithm that first identifies conditional independencies suggested by the data as constraints, and then looks for a structure that best satisfies those. Their book Causation, Prediction, and Search is the most exhaustive source for such things I can think of (Chapters 5 and 6 in particular). A couple of other pointers:

• A nice survey that includes references to further sources on constraint-based algorithms as well as other approaches.
• Lecture notes from Cosma Shalizi that could be a nice way in to the SGS book.