2
$\begingroup$

Suppose we have $3$ random variables, $X, Y, Z$, which verify: $$I(X,Y), I(X,Z), I(Y,Z),$$ where $I(·,·)$ means the variables are independent, and $$D(Y,X \ | \ Z), $$ meaning $Y$ and $X$ are dependent given $Z$.

It is clear not a Bayesian Network neither a Markov Network can model this situation, with the problem mainly coming from the fact that these variables are independent to each other, and thus we cannot place any (directed or undirected) arc between them.

Choosing the empty DAG or graph misses the dependency above. A reasonable Bayesian Network is $Z \to X \leftarrow Y$. This v-structure guarantees it is not losing the information $D(Y,X \ | \ Z)$, but it is assuming $D(Z,X)$ and $D(X,Y)$. At least this can indeed be corrected by using the adequate parameters in the Conditional Probability Tables, i.e. making $P(X \ | \ Z ) = P(X)$.

Now it makes no sense to use a Bayesian Network here, and also all Bayesian Network structure learning algorithms will simply fail (*) building this structure and construct an empty DAG instead.

So, What can be done? Is there a probabilistic graphical model for this situation?


An example toy dataset descriptive of the situation above:

X Y Z
1 1 1
1 0 0
1 1 1
1 0 0
1 1 1
1 0 0
0 1 0
0 0 1
0 1 0
0 0 1
0 1 0
0 0 1

(*) (edited) This is not necessarily true. Actually, it depends on many factors, mainly the depth of the search, meaning it can, indeed, capture the dependency $D(Y,X | Z)$ and thus place the needed arcs. They will still find trouble detecting these types of dependencies in general graphs (this is a toy example)

$\endgroup$
-1
$\begingroup$

The statements that the three variables $ X $, $ Y $ and $ Z $ are pairwise independent is incompatible with the statement that $ X $ and $ Y $ are somehow dependent.

However, the toy dataset seems to indicate some kind of phenomenon where the presence of $ Z $ implies correlation between $ X $ and $ Y $, and the lack of $ Z $ makes them "less correlated" in some sense.

I think you might be slightly confused about the concept of independence, because in this case $ X $, $ Y $, and $ Z $ are certainly dependent. An example of a graphical model that could describe this is:

$$ A \to B $$

where $ A $ is a Bernoulli random variable (which corresponds with the $ Z $ in your model) and $ B $ is a bivariate Bernoulli random variable (i.e. it's a random vector of length 2). In this case, you could define the covariance of $ B_1 $ and $ B_2 $ in terms of $ A $.

$\endgroup$
  • $\begingroup$ "The statements that the three variables $X$, $Y$ and $Z$ are pairwise independent is incompatible with the statement that X and Y are somehow dependent" No, it is not incompatible! Read about conditional dependence and conditional independence. " ...because in this case $X$, $Y$, and $Z$ are certainly dependent... " No, they are not. In this case these three variables are pairwise independent. $\endgroup$ – D1X Jul 11 '18 at 13:26
  • $\begingroup$ I disagree with your first statement. However, I do think you could cook up a Bayesian network that has the property where $ X $ and $ Y $ are independent, but are conditionally dependent given $ Z $. This, of course, means that the only statement you get to make (using your syntax) is $ I(X, Y) $. The network would look like $ X \rightarrow Z \leftarrow Y $. This is counter to your original way of defining the causality, but it would certainly work for your toy dataset. $\endgroup$ – Kevin Li Jul 11 '18 at 14:28
  • $\begingroup$ With what statement precisely you disagree? Yes, of course I can cook up a Bayesian Network for the situation you mention, because that is a v-structure. Also I don't really get what you mean. I did not define causality because causality is not represented in Bayesian Networks. Bayesian Networks only represent independencies. $\endgroup$ – D1X Jan 9 at 9:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.