3
$\begingroup$

Given a joint distribution $P(A,B,C)$, we can compute various marginal distributions. Now suppose: \begin{align} P1(A,B,C) &= P(A) P(B) P(C) \\ P2(A,B,C) &= P(A,B) P(C) \\ P3(A,B,C) &= P(A,B,C) \end{align} Is it true that $d(P1,P3) \geq d(P2,P3)$ where d is the total variation distance?

In other words, is it provable that $P(A,B) P(C)$ is a better approximation of $P(A,B,C)$ than $P(A) P(B) P(C)$ in terms of the total variation distance? Intuitively I think it's true but could not find out a proof.

$\endgroup$
4
  • $\begingroup$ Why introduce $C$ at all? Upon dividing by $P(C)$ your question has nothing to do with $C$--and it looks trivial then. $\endgroup$
    – whuber
    Commented Jul 26, 2013 at 1:16
  • $\begingroup$ @whuber $\frac{P(A,B,C)}{P(C)} = P(A,B|C)$, so $C$ is still involved in $P3$. $\endgroup$
    – took
    Commented Jul 26, 2013 at 2:38
  • $\begingroup$ @took: But $P3$ is just $P(A,B,C)$. So you are just really looking at $P(A,B)$ and $P(A)P(B)$. $\endgroup$ Commented Jul 26, 2013 at 13:30
  • $\begingroup$ @guest43434 I'm looking at the distances between the three distributions, so I think $P3$ is still relevant here. $\endgroup$
    – took
    Commented Jul 26, 2013 at 23:42

1 Answer 1

2
$\begingroup$

I just find the following counter-example. Suppose $A,B,C$ are discrete variables. $A,B$ can each take two values while $C$ can take three values. The joint distribution $P(A,B,C)$ is:

\begin{array}{cccc} A & B & C & P(A,B,C) \\ 1 & 1 & 1 & 0.1/3 \\ 1 & 1 & 2 & 0.25/3 \\ 1 & 1 & 3 & 0.25/3 \\ 1 & 2 & 1 & 0.4/3 \\ 1 & 2 & 2 & 0.25/3 \\ 1 & 2 & 3 & 0.25/3 \\ 2 & 1 & 1 & 0.4/3 \\ 2 & 1 & 2 & 0.25/3 \\ 2 & 1 & 3 & 0.25/3 \\ 2 & 2 & 1 & 0.1/3 \\ 2 & 2 & 2 & 0.25/3 \\ 2 & 2 & 3 & 0.25/3 \\ \end{array}

So the marginal distribution $P(A,B)$ is: \begin{array}{ccc} A & B & P(A,B) \\ 1 & 1 & 0.2 \\ 1 & 2 & 0.3 \\ 2 & 1 & 0.3 \\ 2 & 2 & 0.2 \\ \end{array}

The marginal distributions $P(A), P(B)$ and $P(C)$ are uniform.

So we can compute that: \begin{align} d(P1,P3) &= 0.1 \\ d(P2,P3) &= 0.4/3 \end{align}

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.