# Distance between the product of marginal distributions and the joint distribution

Given a joint distribution $P(A,B,C)$, we can compute various marginal distributions. Now suppose: \begin{align} P1(A,B,C) &= P(A) P(B) P(C) \\ P2(A,B,C) &= P(A,B) P(C) \\ P3(A,B,C) &= P(A,B,C) \end{align} Is it true that $d(P1,P3) \geq d(P2,P3)$ where d is the total variation distance?

In other words, is it provable that $P(A,B) P(C)$ is a better approximation of $P(A,B,C)$ than $P(A) P(B) P(C)$ in terms of the total variation distance? Intuitively I think it's true but could not find out a proof.

• Why introduce $C$ at all? Upon dividing by $P(C)$ your question has nothing to do with $C$--and it looks trivial then.
– whuber
Commented Jul 26, 2013 at 1:16
• @whuber $\frac{P(A,B,C)}{P(C)} = P(A,B|C)$, so $C$ is still involved in $P3$.
– took
Commented Jul 26, 2013 at 2:38
• @took: But $P3$ is just $P(A,B,C)$. So you are just really looking at $P(A,B)$ and $P(A)P(B)$. Commented Jul 26, 2013 at 13:30
• @guest43434 I'm looking at the distances between the three distributions, so I think $P3$ is still relevant here.
– took
Commented Jul 26, 2013 at 23:42

I just find the following counter-example. Suppose $A,B,C$ are discrete variables. $A,B$ can each take two values while $C$ can take three values. The joint distribution $P(A,B,C)$ is:
So the marginal distribution $P(A,B)$ is: \begin{array}{ccc} A & B & P(A,B) \\ 1 & 1 & 0.2 \\ 1 & 2 & 0.3 \\ 2 & 1 & 0.3 \\ 2 & 2 & 0.2 \\ \end{array}
The marginal distributions $P(A), P(B)$ and $P(C)$ are uniform.