Examples of joint probability distribution that cannot be captured by Bayesian Network Can anyone give examples of joint probability distribution that cannot be captured by Bayesian Network?
 A: There does not exists a distribution over a finite number of discrete random variables that cannot be captured by a Bayesian Network:
Given a set of random variables $X_1,\dots,X_n$. Define the parents of $X_i$ to be the variables $X_j$ with $j < i$. This BN encodes no (conditional) independencies at all. Then any given probability distribution $P(X_1,\dots,X_n)$ can be represented by this BN, because of the chain rule holding in general:
$$P(X_1,\dots,X_n) = P(X_1)·P(X_2\mid X_1)·P(X_3\mid X_1,X_2) · … · P(X_n \mid X_1,\dots,X_{n-1})$$
A: I think what you mean is you want an example of a joint distribution with conditional independencies that can't be represented in the BN.  You can always factorize a joint into a product of conditionals using the chain rule, eg: $P(X, Y, Z) = P(Z|X,Y)P(Y|X)P(X)$, which does not assume any independence and can be represented with a directed acyclic network, ie a Bayesian network.  The question is what conditional independencies are there between $X, Y$, and $Z$?
There is a type of 3-way interaction, I've seen it called "uniform association".  The following factorization of the joint: $P(X, Y, Z) = P(X, Y) P(Y, Z) P (Z, X) P(X) P(Y) P(Z)$.  This is not very intuitive, I know, other than to say that or any value of one variable, the association between the other two variables remains constant.
The bivariate probability distributions in that factorization cannot be factorized into conditional probabilities without introducing cycles, thus this factorization cannot be represented with a directed acyclic network.
To model this type of independence, you could use a generalized linear model.
Just found a reference: see page 16 of these notes from Princeton (pdf).
A: Take a DAG graph with with N random variables. Using the chain rule you can encode all independencies in that particular graph.
