0
$\begingroup$

Consider the following Bayes network of random variables on some probability space:

Example Bayes Graph

The local Markov property asserts that any variable is independent of its non-descendants given its parents. Here, $X$ has no parents, and $Y$ is a non-descendant of $X$. Thus $X$ is independent of $Y$.

A Markov blanket of the variable $X$ is any subset of other variables that "contains all the information needed to infer $X$." The Markov boundary is the smallest such subset, i.e. the Markov blanket with "no redundant information."

For a Bayes network, the Markov boundary of $X$ consists of its parents, its children, and the other parents of its children (its "spouses"). For our $X$, this is $\{Y,Z\}$.

I am perplexed that $X$ can be independent of $Y$ and yet $Y$ is still in its Markov boundary. The implication is that $Y$ provides a critical piece of information about $X$ when trying to decouple it from the rest of the network. I.e., I cannot assume $X$ is independent of $\{U,V\}$ unless I condition on both $Z$ and $Y$. And yet, somehow, $X$ is "independent" of $Y$.

This leads me to believe that maybe $Y$ is redundant, and that the Markov boundary here is really just $Z$. But then I look at any source explaining Markov boundaries and they provide explanations like this that have the same issue:

Typical Markov boundary explanation

It is always true that $A$ given its parents is independent of its spouses, because the spouses are always non-descendants. Since the Markov boundary always includes the parents and children, how could the spouses have any additional information to provide?

I am struggling to understand the role of spouses in the Markov boundary, because it leads me to the following statement which I thought was definitely invalid: $$ p(X|Y) = p(X)\ \ \ \ \text{while}\ \ \ \ p(X|Y,Z) \neq p(X|Z) \tag{1} $$

Question:

How is the above Statement (1) valid?

If it's not, then doesn't $p(X|Y,Z) = p(X|Z)$ imply $Y$ isn't in the Markov boundary?

$\endgroup$

1 Answer 1

0
$\begingroup$

Addressing your question at the end, it is in fact possible for statement (1) to be true: $X$ and $Y$ may be unconditionally independent but conditionally dependent given $Z$. In the case that $Z$ is observed, the $X \to Z \leftarrow Y$ subgraph is referred to as an "activated v-structure". For an intuitive example, let $X, Y, Z$ be indicator random variables as follows:

  • $X = 1$ iff my bus was late.
  • $Y = 1$ iff I overslept.
  • $Z = 1$ iff I am late to work.

In general $X$ and $Y$ will be unconditionally independent: knowing whether I overslept tells you nothing about whether my bus was late, in the absence of further information.

If, however, you know that I am late to work ($Z=1$), then $X$ and $Y$ become conditionally dependent: knowing that I didn't oversleep should increase your belief that my bus was late, for example.

I encourage you to try and come up with a concrete joint distribution over $X,Y,Z$ that respects the graph structure $X \to Z \leftarrow Y$ while also encoding the conditional dependence described above. (If you get stuck, check out the example from Wikipedia's article on conditional dependence.)

$\endgroup$
2
  • $\begingroup$ That pretty much answers it! Your example made a lot of sense. So since Statement (1) is valid, using your intuition I can see why $Y$ is in the Markov boundary of $X$ and is still unconditionally independent of $X$. I think the layman way of describing the Markov boundary as "all the information needed with nothing redundant" is fair. In the end my fallacy was thinking that unconditional independence equates to "cannot be related under any circumstances." I'm very glad to have cleared that up. $\endgroup$
    – jnez71
    Dec 27, 2020 at 10:41
  • $\begingroup$ For posterity, the reason I asked this was because I had an example where Statement (1) was true (but I thought paradoxical). I had a real-valued discrete-time random walk $q_{t+1} = q_t + w_t$ where $w$ is an IID Gaussian process. By construction, $p(w_t | q_t) = p(w_t)$. However, $q_t$ and $q_{t+1}$ completely specify $w_t$ as $p(w_t | q_t, q_{t+1}) = \delta_{q_{t+1} - q_t} \neq p(w_t | q_{t+1}) = \mathcal{N}$. Relating this to my posted question's abstraction, $(w_t, q_t, q_{t+1}) \iff (X, Y, Z)$. $\endgroup$
    – jnez71
    Dec 27, 2020 at 19:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.