5
$\begingroup$

There are three random variables $A$, $B$ and $C$. If the variables $A$ and $B$ were independent, their marginal joint distribution would be given by $$ P(A,B) = P(A)P(B) $$

For example, given the discrete probability distributions of $A = \{ A_1, A_2 \}$:

A1 0.6
A2 0.4

and of $B = \{B_1, B_2 \}$:

B1 0.4
B2 0.6

The joint probability distribution would then be $P(A,B)=P(A)P(B)$:

A1 B1 0.24
A1 B2 0.36
A2 B1 0.16
A2 B2 0.24

However, If the variables are not independent, and we know the marginal joint distributions of $P(A,C)$ and $P(B,C)$, e.g.

A1 C1  0.4       B1 C1  0.1
A1 C2  0.2       B1 C2  0.3
A2 C1  0.2       B2 C1  0.5
A2 C2  0.2       B2 C2  0.1

Is that enough to build the joint distribution of $P(A,B,C)$? And, additionally, the marginal joint distribution of $P(A,B)$?

$\endgroup$
5
  • 1
    $\begingroup$ You already have the joint distribution of P(AB)=P(A)P(B). I don't understand why are you trying to use C to obtain P(AB) when you already have that from your first step? Also, P(AB) is not equal to P(A)P(B|A). $\endgroup$ Nov 7, 2018 at 18:08
  • $\begingroup$ Please explain to us how $C$ "gives us more information:" what does it represent? What do the entries in your last table mean? $\endgroup$
    – whuber
    Nov 7, 2018 at 18:21
  • $\begingroup$ The P(AB)=P(A)P(B) only in the case of the events being independent. Their respective joint distributions with C show this not to be the case. $\endgroup$ Nov 7, 2018 at 19:12
  • 1
    $\begingroup$ I reopened the question despite the fact the criteria in your example are inconsistent. For instance, you are asserting that the probability of C1 is $0.4+0.1=0.5$ while also asserting it equals $0.1+0.5=0.6.$ The question might be clearer to some readers if the example actually had a solution. That might help everyone see that in general there are many solutions. $\endgroup$
    – whuber
    Nov 7, 2018 at 20:09
  • $\begingroup$ Thanks, I updated the probabilities so they should be consistent. $\endgroup$ Nov 7, 2018 at 20:16

2 Answers 2

1
$\begingroup$

In the general case, no. The marginals of $P(A,C)$ and $P(B,C)$ are enough to determine the full joint distribution of $P(A,B,C)$ only in the case of conditional independence: $$ P(A \mid B,C) = P(A\mid C) $$ and equivalently: $$ P(B \mid A,C) = P(B\mid C). $$


Starting with the full joint probability $P(A,B,C)$. Given the formula for joint probability $$P(X,Y)=P(X\mid Y)P(Y),$$ we get:

$$ P(A,B,C)=P(A,C)P(B\mid A,C). $$

If the events $A$ and $B$ are not conditionally independent given $C$, now we would need to know the distribution of $P(B \mid A,C)$, which would be enough to form the full joint distribution.

If we assume the probability of B given C be independent of A, the formula simplifies to: $$ P(A,B,C)=P(A,C)P(B\mid C). $$

Using the example probabilities, e.g. $$ P(A_1,B_1,C_1) = P(A_1,C_1)P(B_1\mid C_1) = 0.4 \cdot \frac{0.1}{0.1+0.5} = \frac{2}{30}, $$

where $P(B_1 \mid C_1)$ is the conditional probability of observing $B_1$ after observing $C_1$.

The full table of probabilities is then:

A1 B1 C1    4/60
A1 B1 C2    9/60
A1 B2 C1   20/60
A1 B2 C2    3/60
A2 B1 C1    2/60
A2 B1 C2    9/60
A2 B2 C1   10/60
A2 B2 C2    3/60

And the joint distribution of A, B (summing over C):

A1 B1   13/60   0.2166...
A1 B2   23/60   0.3833...
A2 B1   11/60   0.1833...
A2 B2   13/60   0.2166...

We can see that $A$ and $B$ are not independent -- but they are still conditionally independent given $C$! If this is not true, the full joint distribution cannot be built from marginal joint distributions like this.

$\endgroup$
6
  • 3
    $\begingroup$ So that readers don't leave with the wrong impression that this kind of information determines the joint distribution, you really ought to point out how unusual your set of conditions is: it permits a unique answer, whereas in general there will be a two-parameter family of solutions. $\endgroup$
    – whuber
    Nov 7, 2018 at 23:11
  • $\begingroup$ Hi @Huber, so are you suggesting that for any two marginals P(A,B) and P(B,C) there are an infinite number of joint distributions P(A,B,C)? Do you have a reference for this fact? $\endgroup$
    – Apprentice
    Apr 19, 2022 at 15:07
  • 1
    $\begingroup$ @Apprentice I edited my answer to take into account his comment -- specifically the first part, that they need to be (conditionally) independent. Only then you can get the joint probability by multiplication. There are not infinite number of joint distributions, but you would need the extra info about their relationship to get to the single joint distribution. $\endgroup$ Apr 21, 2022 at 9:03
  • 2
    $\begingroup$ The number of solutions is infinite generally. That is because as soon as you have two distinct solutions, any mixture of them will be a solution, too, and all those mixtures will be different. $\endgroup$
    – whuber
    Apr 21, 2022 at 12:30
  • $\begingroup$ @whuber how do you prove that exist two distinct solutions then? $\endgroup$
    – Apprentice
    Apr 21, 2022 at 13:53
1
$\begingroup$

For the 2x2x2 example case we can have the same joint distribution for

$$P'(000) = P(000) + x \\ P'(001) = P(001) - x \\ P'(010) = P(010) - x \\ P'(011) = P(011) + x \\ P'(100) = P(100) - x \\ P'(101) = P(101) + x \\ P'(110) = P(110) + x \\ P'(111) = P(111) - x$$

In this case for any marginal, it will be a sum of two of the cases where one has -x and the other +x, leaving everything unchanged.


Alternative reasoning: You can consider the number of parameters.

In your example:

  • The marginal conditions involve two times three independent parameters
  • The joint distribution has seven independent parameters.

This indicates that the marginal distributions does not fix the joint distribution.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.