Can anyone give examples of joint probability distribution that cannot be captured by Bayesian Network?
$\begingroup$
$\endgroup$
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
7
There does not exists a distribution over a finite number of discrete random variables that cannot be captured by a Bayesian Network:
Given a set of random variables $X_1,\dots,X_n$. Define the parents of $X_i$ to be the variables $X_j$ with $j < i$. This BN encodes no (conditional) independencies at all. Then any given probability distribution $P(X_1,\dots,X_n)$ can be represented by this BN, because of the chain rule holding in general:
$$P(X_1,\dots,X_n) = P(X_1)·P(X_2\mid X_1)·P(X_3\mid X_1,X_2) · … · P(X_n \mid X_1,\dots,X_{n-1})$$
-
$\begingroup$ Your example is "a BN" that can not represent a joint probability, but Not "a joint probability" that can not be captured by a BN. $\endgroup$– TimCommented Mar 11, 2015 at 20:04
-
-
$\begingroup$ What is your center variable? If you want nothing to be independent, it has to be always observed. I would have set it up with all the arrows pointing out, and the center variable representing the universe of hypothetical causes of everything. Anyway, seems very Buddhist haha. $\endgroup$– Neil GCommented Mar 11, 2015 at 20:40
-
$\begingroup$ @Neil You're right, I'm wrong. $\endgroup$ Commented Mar 11, 2015 at 21:16
-
$\begingroup$
$\endgroup$
2
I think what you mean is you want an example of a joint distribution with conditional independencies that can't be represented in the BN. You can always factorize a joint into a product of conditionals using the chain rule, eg: $P(X, Y, Z) = P(Z|X,Y)P(Y|X)P(X)$, which does not assume any independence and can be represented with a directed acyclic network, ie a Bayesian network. The question is what conditional independencies are there between $X, Y$, and $Z$?
There is a type of 3-way interaction, I've seen it called "uniform association". The following factorization of the joint: $P(X, Y, Z) = P(X, Y) P(Y, Z) P (Z, X) P(X) P(Y) P(Z)$. This is not very intuitive, I know, other than to say that or any value of one variable, the association between the other two variables remains constant.
The bivariate probability distributions in that factorization cannot be factorized into conditional probabilities without introducing cycles, thus this factorization cannot be represented with a directed acyclic network.
To model this type of independence, you could use a generalized linear model.
Just found a reference: see page 16 of these notes from Princeton (pdf).
-
$\begingroup$ Yes, the (conditional) independence structures that can be represented by BNs and by Markov networks differ. There exist MNs for which no "equivalent" BN exists and vice versa (see Judea Pearl's Probablistic Reasoning in Intelligent Systems for proofs and examples). $\endgroup$ Commented Mar 13, 2015 at 8:27
-
$\begingroup$ @ziggystar good point. The factorization I describe is a "diamond shape" as a Markov network. I think Pearl (or Koller?) points out a diamond can't be represented as a DAG $\endgroup$ Commented Apr 11, 2015 at 12:55
$\begingroup$
$\endgroup$
Take a DAG graph with with N random variables. Using the chain rule you can encode all independencies in that particular graph.
answered Mar 11, 2015 at 23:06