# How to derive marginal probability in hybrid bayesian networks?

Suppose I am having the following hybrid network

where $A$ is boolean and $D, E \& G$ are continuous random variables, Also suppose the following:

       p(A)
True   0.0948
False  0.9052


And

          µ(D)   var(D)
A=True    7.3555    0.27055
A=False   12.3433   0.10864


And $E$ is described as a linear gaussian given its parent $D$ with $$N(0.995 + 2.351 D; 13.732)$$ And $G$ is described as a conditional linear gaussian under all possible configurations of $A$ given its continuous parents $D\&E$ $$N(4.9491 + 1.002E + 2.252D; 11.356) \text{ when } A = True$$ $$N(2.384 + 1.0001E + 4.066D; 47.008) \text{ when } A = False$$

Now I am really confused about how to calculate the following marginal probabilities: $$P(G ≥ 30)$$ $$P(G ≥ 30 | E < 20)$$ $$P(G ≥ 30 | E < 20 , A=True)$$

I have pretty much surfed the whole internet and I didn't find any clear example on how to calculate them all the examples are on discrete bayesian networks, so any help would be appreciated as stats exchange is my last resort now.

• Could you clarify what you mean by N(0.995 + 2.351D; 13.732) and mu = 28.813? I read that as two different specifications for the mean: i) 0.995 + 2.351D, and ii) 28.813. Ditto for the specification for G below it. – jtobin Sep 19 '15 at 0:34
• oh I can see where the confusion is coming from.. $0.995 + 2.315D$ is the linear mean of $E$ depending on $D$ where 0.995 is the intercept and 2.315 is the coefficent while 28.813 is the original mean of $E$..I think its better to edit the question and remove them to prevent any other future confessions – m.awad Sep 19 '15 at 0:42

I'll ignore the specific coefficients and play integral notation a bit loose in most places so as to avoid clutter.

Take your first question. You want to evaluate the following integral:

\begin{equation*} \mathbb{P}(G \geq 30) = \int \mathcal{I}_{G \geq 30} \, p(G, A, D, E) \, d\mathbb{P} \end{equation*}

An easy way to break this up is to just split it across $A$ right away:

\begin{align*} P(G \geq 30) & = \int \mathcal{I}_{G \geq 30} \, p(G, D, E \, | \, A) p(A)\, dP \\ & = p_{A} \int \mathcal{I}_{G \geq 30} p(G, D, E \, | \, A) dP + (1 - p_{A}) \int \mathcal{I}_{G \geq 30} p(G, D, E \, | \, \neg A) dP \end{align*}

Now on the left you have the case where $A$ is true, and on the right the case where $A$ is false.

Now take the $p(G, D, E \, | \cdot)$ terms. Given the structure of your model you can factorize these joint distributions appropriately:

\begin{align*} p(G, D, E \, | \, A) & = p(G \, | \, D, E, A) p(E \, | \, D, A) p(D \, | \, A) \\ p(G, D, E \, | \, \neg A) & = p(G \, | \, D, E, \neg A) p(E \, | \, D, \neg A) p(D \, | \, \neg A) \end{align*}

You can then substitute in the information you have in order to evaluate the integral, i.e. by substituting in the requisite density functions that you already have and integrating over everything.

For your second question you'll want to integrate over $P(G \, | \, E)$. So you can follow the same kind of procedure; first to calculate $P(G, E)$ \begin{align*} p(G, E) & = \int_{A, D} p(G, A, D, E) dP_{A, D} \\ & = p_{A} \int_{D} p(G, D, E \, | \, A) dP_{D} + (1 - p_{A}) \int_{D} p(G, D, E \, | \, \neg A) dP_{D} \\ & = p_{A} \int_{D} p(G, E \, | \, D, A) p(D \, | \, A) dP_{D} + (1 - p_{A}) \int_{D} p(G, E \, | \, D, \neg A) p(D \, | \, \neg A) dP_{D} \end{align*}

and then more of the same to calculate $P(E)$, giving you $P(G \, | \, E) = \frac{P(G, E)}{P(E)}$. Then $P(G \geq 30 \, | \, E < 20)$ would be

\begin{equation*} P(G \geq 30 \, | \, E < 20) = \int \mathcal{I}_{G \geq 30, E < 20} p(G \, | \, E) dP. \end{equation*}

The third question is the simplest; it's more of the same, except you'll only need to consider the case where $A$ is true.

EDIT:

Actually evaluating the integrals requires some legwork.

For a single example, here is the expanded expression for $\int \mathcal{I}_{G \geq 30} p(G, D, E | A)dP$ given in terms of the individual conditional densities $p(G | D, E, A)$, $p(E | D, A)$, and $p(D | A)$:

\begin{align*} \int \mathcal{I}_{G \geq 30} p(G, D, E | A)dP = & \int_{30}^{\infty} \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} \frac{1}{\sqrt{2\pi(0.27055)}} \exp \left\{ - \frac{(d - 7.355)^2}{2(0.27055)} \right\} \\ & \frac{1}{\sqrt{2\pi(13.732)}} \exp \left\{ - \frac{(e - (0.995 + 0.2351d))^2}{2(13.732)} \right\} \\ & \frac{1}{\sqrt{2\pi(11.356)}} \exp \left\{ - \frac{(g - (4.9491 + 1.002e))^2}{2(11.356)} \right\} dd \, de \, dg \end{align*}

You'll likely want to use a numerical or approximate integration algorithm in practice.

• Thanks for your help...your answer is sufficiently detailed and it helped me alot to clarify things...but I still don't get the integral part that's why I posted the questions with a clear example hoping that someone will clear that noise for me, the thing that I don't get is how should I integrate over it should I use the CDF? and how do I even do it If I don't have the mean (the mean is still a linear combination of the node's parent), if you can elaborate more on that part or even point me to some link that explains it that would be very kind of you. – m.awad Sep 19 '15 at 12:09
• I think that there is a tiny mistake here you said $P(G,D,E|A) = P(G|D,E,A)P(E|D,A)P(D|A)$ but I think it should be $P(G,D,E|A) = P(G|D,E,A)P(E|D)P(D|A)$ as $E$ is independent of $A$, don't you agree? – m.awad Sep 23 '15 at 13:33
• @m.awad Yes, $E$ is conditionally independent of $A$ given $D$, so $P(E|D, A) = P(E|D)$. – jtobin Sep 24 '15 at 4:02