Can the marginal distributions of A,C and B,C be used to build joint distribution of A and B?

Question

There are three random variables $A$, $B$ and $C$. If the variables $A$ and $B$ were independent, their marginal joint distribution would be given by $$ P(A,B) = P(A)P(B) $$

For example, given the discrete probability distributions of $A = \{ A_1, A_2 \}$:

A1 0.6
A2 0.4

and of $B = \{B_1, B_2 \}$:

B1 0.4
B2 0.6

The joint probability distribution would then be $P(A,B)=P(A)P(B)$:

A1 B1 0.24
A1 B2 0.36
A2 B1 0.16
A2 B2 0.24

However, If the variables are not independent, and we know the marginal joint distributions of $P(A,C)$ and $P(B,C)$, e.g.

A1 C1  0.4       B1 C1  0.1
A1 C2  0.2       B1 C2  0.3
A2 C1  0.2       B2 C1  0.5
A2 C2  0.2       B2 C2  0.1

Is that enough to build the joint distribution of $P(A,B,C)$? And, additionally, the marginal joint distribution of $P(A,B)$?

Answer 1

In the general case, no. The marginals of $P(A,C)$ and $P(B,C)$ are enough to determine the full joint distribution of $P(A,B,C)$ only in the case of conditional independence: $$ P(A \mid B,C) = P(A\mid C) $$ and equivalently: $$ P(B \mid A,C) = P(B\mid C). $$

---

Starting with the full joint probability $P(A,B,C)$. Given the formula for joint probability $$P(X,Y)=P(X\mid Y)P(Y),$$ we get:

$$ P(A,B,C)=P(A,C)P(B\mid A,C). $$

If the events $A$ and $B$ are not conditionally independent given $C$, now we would need to know the distribution of $P(B \mid A,C)$, which would be enough to form the full joint distribution.

If we assume the probability of B given C be independent of A, the formula simplifies to: $$ P(A,B,C)=P(A,C)P(B\mid C). $$

Using the example probabilities, e.g. $$ P(A_1,B_1,C_1) = P(A_1,C_1)P(B_1\mid C_1) = 0.4 \cdot \frac{0.1}{0.1+0.5} = \frac{2}{30}, $$

where $P(B_1 \mid C_1)$ is the conditional probability of observing $B_1$ after observing $C_1$.

The full table of probabilities is then:

A1 B1 C1    4/60
A1 B1 C2    9/60
A1 B2 C1   20/60
A1 B2 C2    3/60
A2 B1 C1    2/60
A2 B1 C2    9/60
A2 B2 C1   10/60
A2 B2 C2    3/60

And the joint distribution of A, B (summing over C):

A1 B1   13/60   0.2166...
A1 B2   23/60   0.3833...
A2 B1   11/60   0.1833...
A2 B2   13/60   0.2166...

We can see that $A$ and $B$ are not independent -- but they are still conditionally independent given $C$! If this is not true, the full joint distribution cannot be built from marginal joint distributions like this.

Answer 2

For the 2x2x2 example case we can have the same joint distribution for

$$P'(000) = P(000) + x \\ P'(001) = P(001) - x \\ P'(010) = P(010) - x \\ P'(011) = P(011) + x \\ P'(100) = P(100) - x \\ P'(101) = P(101) + x \\ P'(110) = P(110) + x \\ P'(111) = P(111) - x$$

In this case for any marginal, it will be a sum of two of the cases where one has -x and the other +x, leaving everything unchanged.

---

Alternative reasoning: You can consider the number of parameters.

In your example:

• The marginal conditions involve two times three independent parameters
• The joint distribution has seven independent parameters.

This indicates that the marginal distributions does not fix the joint distribution.