# Calculating joint probability and covariance given conditional probabilities

If we have the tables of conditionals $p(x|y)$ and $p(y|x)$, how can we calculate the joint probability $p(x, y)$? (Let us assume $x$ and $y$ are binary variables).

Using these tables, how can we calculate the covariance of $x$ and $y$? Covariance of two variables are calculated using $E[xy]-E[x]E[y]$, but I am not sure how to evaluate this considering the probabilities.

To clarify further, let me show the question as follows:

• Can you give an example of table of conditional $p(x|y)$? I suspect that $p(y)$ must given also and then $p(x,y)=p(x|y)p(y)$ and calculation of covariance is trivial. – mpiktas Oct 7 '13 at 12:25
• @mpiktas $p(y)$ is not given. I have added example tables, but I am not sure if these numeric values are correct since I cannot calculate correctly. – groove Oct 8 '13 at 10:24

I think the given probabilities are wrong.

Using $p(x|y)p(y)=p(y|x)p(x)=p(x,y)$, we can write $p(x,y)$ as follows:

$p(x=0, y=0)=0.2p(y=0)=0.7p(x=0)$ [1]

$p(x=1, y=0)=0.8p(y=0)=0.4p(x=1)$ [2]

$p(x=0, y=1)=0.6p(y=1)=0.3p(x=0)$ [3]

$p(x=1, y=1)=0.4p(y=1)=0.6p(x=1)$ [4]

Lets take the first two:

$2p(y=0)=7p(x=0)$

$2p(y=0)=p(x=1)=(1-p(x=0))$ where $p(x=1) = 1-p(x=0)$

we get $p(x=0)=1/8$, $p(x=1)=7/8$, $p(y=0)=7/16$ and $p(y=1)=9/16$.

If we put these in [3], we get $9=1$.

• +1 Well done! Each diagram determines a linear relation between $\Pr(x=1)$ and $\Pr(y=1)$. The two diagrams therefore should be consistent with each other, but you have shown they are not. – whuber Oct 8 '13 at 20:05
• @whuber Actually I remember a calculation method by proportioning and scaling and not by calculating the exact marginals, but this question confused me. Maybe that faster way is correct although I could not prove it in this question. – groove Oct 8 '13 at 20:13

Given such information it is possible to calculate the marginal distributions and then the joint distribution follows.

For a discrete variable $\sum_{i=1}^nP(X=x_{i}|A)=1$, hence we immediately can fill in the missing values in the conditional distribution tables.

• $P(x=1|y=0)=0.8$,
• $P(x=0|y=1)=0.6$,
• $P(y=0|x=0)=0.7$
• $P(y=1|x=1)=0.6$

Now

$$P(x=0|y=0)=\frac{P(x=0,y=0)}{P(y=0)}=\frac{P(y=0|x=0)P(x=0)}{P(y=0)}=0.2$$

giving us

$$\frac{P(x=0)}{P(y=0)}=\frac{0.2}{0.7}$$

then similarly

$$P(x=1|y=0)=\frac{P(x=1,y=0)}{P(y=0)}=\frac{P(y=0|x=1)P(x=1)}{P(y=0)}=0.8$$

giving us

$$\frac{P(x=1)}{P(y=0)}=\frac{0.8}{0.4}$$

Since $P(x=1)+P(x=0)=1$ we get

$$\frac{1}{P(y=0)}=\frac{0.2}{0.7}+\frac{0.8}{0.4}$$

And we have $P(y=0)$ and hence $P(y=1)$. $P(x=0)$ and $P(x=1)$ then follows directly. So we have marginal distributions and conditional distributions and we can complete the joint distribution table.

• This answer is great. But I think the given tables are wrong, I will try to explain it as a new answer. Would you like to check it and add something about the covariance or should I just select your answer as the accepted? – groove Oct 8 '13 at 19:36