2
$\begingroup$

I am trying to derive the formula for the conditional distribution for a variable in a Bayesian network: $$p(x_j|x_{-j})=p(x_j|x_{pa(j)})\prod_{k\in ch(j)}p(x_k|x_{pa(k)})$$ I understand D-separation and why these variables are the only ones involved, but I just can't seem to get the derivation right. In particular, the integral in the denominator must evaluate to 1, and I don't see why. Anyway, here's my attempt:

$$p(x_j|x_{-j})=\frac{p(x)}{p(x_{-j})}=\frac{\prod_i p(x_i|x_{pa(i)})}{\int \prod_i p(x_i|x_{pa(i)})dx_j} $$ $$=\frac{p(x_j|x_{pa(j)})\prod_{i\neq j} p(x_i|x_{pa(i)})}{\prod_{k\neq j, k\notin ch(j)} p(x_k|x_{pa(k)})\int p(x_j|x_{pa(j)})\prod_{l\in ch(j)}p(x_l|x_{pa(l)})dx_j}$$ $$ \frac{p(x_j|x_{pa(j)})\prod_{k\in ch(j)}p(x_k|x_{pa(k)})}{\int p(x_j|x_{pa(j)})\prod_{l\in ch(j)}p(x_l|x_{pa(l)})dx_j}$$

In order to get the right answer, the integral must equal 1, but I don't see why that should be.

$\endgroup$
2
  • $\begingroup$ I am confused as to what you are trying to do. $p(x_{j}|x_{-j}) = p(x_{j}|x_{pa(j)}) $ in a Bayesian network Where did you get the $\Pi$ term from ?? $\endgroup$
    – Sid
    Jul 22, 2014 at 5:31
  • $\begingroup$ second equation on pg 23. If you condition on the children, you have to take them into account as well, so you can't just write what you have there. $\endgroup$ Jul 22, 2014 at 11:41

1 Answer 1

0
$\begingroup$

A little late but the short answer is I don't think the denominator is $1$ in general. I believe some online notes mean to put a proportionality symbol but accidentally write $=$.

I think your proof is correct. I tried deriving the answer in a different way which I have written below but it might be wrong.

A proof of the statement can be seen as follows. Using an understanding of $d$-separation, it can be shown that $p(x_i | x_{-1}) = p(x_i | MB(x_i))$ where $MB(x_i)$ is the Markov blanker of $x_i$. For notational convenience we can define $\text{Pa}_{x_i}$ to be the parents of $x_i$, $\text{Ch}_{x_i}$ to be the direct children of $x_i$, and $\text{Pa}_{\text{Ch}_{x_i}}$ to be the set of parents of any child of $x_i$.

Conditioned on everything, the only nodes that aren't independent from $x_i$ are the nodes in the Markov blanket or equivalently, the nodes in $\text{Pa}_{x_i} \cup \text{Ch}_{x_i} \cup \text{Pa}_{\text{Ch}_{x_i}}$.

Therefore, by Bayes' rule

$$ \begin{align*} p(x_i | x_{-i}) &= p(x_i | MB(x_i)) \\ &= p(x_i | \text{Pa}_{x_i}, \text{Ch}_{x_i}, \text{Pa}_{\text{Ch}_{x_i}}) \\ &= \frac{p(\text{Ch}_{x_i} | x_i, \text{Pa}_{x_i}, \text{Pa}_{\text{Ch}_{x_i}})p( x_i | \text{Pa}_{x_i}, \text{Pa}_{\text{Ch}_{x_i}})}{\int p(\text{Ch}_{x} | x, \text{Pa}_{x}, \text{Pa}_{\text{Ch}_{x}})p( x | \text{Pa}_{x}, \text{Pa}_{\text{Ch}_{x}}) \mathop{dx}} \\ &= \frac{p(\text{Ch}_{x_i} | \text{Pa}_{\text{Ch}_{x_i}})p( x_i | \text{Pa}_{x_i}) }{\int p(\text{Ch}_{x} | \text{Pa}_{\text{Ch}_{x}})p( x | \text{Pa}_{x}) \mathop{dx} }\\ &= \frac{p( x_i | \text{Pa}_{x_i})\prod_{x_j \in \text{Ch}_{x_i}} p(x_j | \text{Pa}_{x_j})}{\int p( x | \text{Pa}_{x})\prod_{x_j \in \text{Ch}_{x}} p(x_j | \text{Pa}_{x_j}) \mathop{dx}}. \end{align*} $$

where the third line is Bayes' rule, and the last two lines use conditional independence properties. There are a few small issues with this proof in terms of the children and parents of children not being disjoint but this is kind of an intuition. You can formalise it by using a reverse topological ordering of the variables and then you will basically get your proof.

This might be wrong because I haven't been able to actually find a proof of this derivation anywhere online, but I am fairly certain that the denominator in your question is not always $1$. For example, see Koller and Friedman pg. 513.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.