There are three random variables $A$, $B$ and $C$. If the variables $A$ and $B$ were independent, their marginal joint distribution would be given by $$ P(A,B) = P(A)P(B) $$

For example, given the discrete probability distributions of $A = \{ A_1, A_2 \}$:

A1 0.6
A2 0.4

and of $B = \{B_1, B_2 \}$:

B1 0.4
B2 0.6

The joint probability distribution would then be $P(A,B)=P(A)P(B)$:

A1 B1 0.24
A1 B2 0.36
A2 B1 0.16
A2 B2 0.24

However, If the variables are not independent, and we know the marginal joint distributions of $P(A,C)$ and $P(B,C)$, e.g.

A1 C1  0.4       B1 C1  0.1
A1 C2  0.2       B1 C2  0.3
A2 C1  0.2       B2 C1  0.5
A2 C2  0.2       B2 C2  0.1

Is that enough to build the joint distribution of $P(A,B,C)$? And, additionally, the marginal joint distribution of $P(A,B)$?

  • 1
    $\begingroup$ You already have the joint distribution of P(AB)=P(A)P(B). I don't understand why are you trying to use C to obtain P(AB) when you already have that from your first step? Also, P(AB) is not equal to P(A)P(B|A). $\endgroup$ – StatsStudent Nov 7 '18 at 18:08
  • $\begingroup$ Please explain to us how $C$ "gives us more information:" what does it represent? What do the entries in your last table mean? $\endgroup$ – whuber Nov 7 '18 at 18:21
  • $\begingroup$ The P(AB)=P(A)P(B) only in the case of the events being independent. Their respective joint distributions with C show this not to be the case. $\endgroup$ – Sakari Cajanus Nov 7 '18 at 19:12
  • 1
    $\begingroup$ I reopened the question despite the fact the criteria in your example are inconsistent. For instance, you are asserting that the probability of C1 is $0.4+0.1=0.5$ while also asserting it equals $0.1+0.5=0.6.$ The question might be clearer to some readers if the example actually had a solution. That might help everyone see that in general there are many solutions. $\endgroup$ – whuber Nov 7 '18 at 20:09
  • $\begingroup$ Thanks, I updated the probabilities so they should be consistent. $\endgroup$ – Sakari Cajanus Nov 7 '18 at 20:16

In the general case, no. The marginals of $P(A,C)$ and $P(B,C)$ are enough to determine the full joint distribution of $P(A,B,C)$ only in the case of conditional independence: $$ P(A \mid B,C) = P(A\mid C) $$ and equivalently: $$ P(B \mid A,C) = P(B\mid C). $$

Starting with the full joint probability $P(A,B,C)$. Given the formula for joint probability $$P(X,Y)=P(X\mid Y)P(Y),$$ we get:

$$ P(A,B,C)=P(A,C)P(B\mid A,C). $$

If the events $A$ and $B$ are not conditionally independent given $C$, now we would need to know the distribution of $P(B \mid A,C)$, which would be enough to form the full joint distribution.

If we assume the probability of B given C be independent of A, the formula simplifies to: $$ P(A,B,C)=P(A,C)P(B\mid C). $$

Using the example probabilities, e.g. $$ P(A_1,B_1,C_1) = P(A_1,C_1)P(B_1\mid C_1) = 0.4 \cdot \frac{0.1}{0.1+0.5} = \frac{2}{30}, $$

where $P(B_1 \mid C_1)$ is the conditional probability of observing $B_1$ after observing $C_1$.

The full table of probabilities is then:

A1 B1 C1    4/60
A1 B1 C2    9/60
A1 B2 C1   20/60
A1 B2 C2    3/60
A2 B1 C1    2/60
A2 B1 C2    9/60
A2 B2 C1   10/60
A2 B2 C2    3/60

And the joint distribution of A, B (summing over C):

A1 B1   13/60   0.2166...
A1 B2   23/60   0.3833...
A2 B1   11/60   0.1833...
A2 B2   13/60   0.2166...

We can see that $A$ and $B$ are not independent -- but they are still conditionally independent given $C$! If this is not true, the full joint distribution cannot be built from marginal joint distributions like this.

  • $\begingroup$ So that readers don't leave with the wrong impression that this kind of information determines the joint distribution, you really ought to point out how unusual your set of conditions is: it permits a unique answer, whereas in general there will be a two-parameter family of solutions. $\endgroup$ – whuber Nov 7 '18 at 23:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.