For two random variables $A$ and $B$, if we know $\mathbb{P}(A|B)$ for all possible values of $A$ given all possible values of $B$ respectively, and $\mathbb{P}(B|A)$ for all possible values of $B$ given all possible values of $A$ respectively, can we find $\mathbb{P}(A,B)$ for all possible values of $(A, B)$? If we can find them, and if we treat $\mathbb{P}(A,B)$ collectively as a function of values of $A$ and $B$, is such function unique?


I've found something pertaining to my question, but I'm not sure if it's correct. I'll present it below. Please point out any error in it. Much appreciated. It's in terms of the probability density functions.

First of all, let's set up the problem. Suppose we're given the conditional PDF's $f(x|y)$ and $f(y|x)$ for all possible values of $x$ and $y$. They are each a function supported on $\mathcal{X}\times\mathcal{Y} \subset\mathbb{R}^2$, where $\mathcal{X}$ and $\mathcal{Y}$ are the set of all possible values of $x$ and $y$ respectively. We want to find the joint PDF $f(x,y)$ also supported on $\mathcal{X}\times\mathcal{Y}$. Note that since each of them is a PDF, $\int_{\mathcal{X}}f(x|y)=\int_{\mathcal{Y}}f(y|x)=\iint_{\mathcal{X}\times\mathcal{Y}}f(x,y)=1.$ (I omitted $\text{d}x$ and $\text{d}y$ for succinctness.)

Now, given an arbitrary pair of conditional PDF's, we're not guaranteed to find the joint PDF. This is called the compatibility of the conditional distributions. We can see this fact from the chain rule $f(x,y)=f(x|y)f(y)=f(y|x)f(x)$. It needs a little bit of explicit explanation, as follows. The chain rule suggests that $\frac{f(x|y)}{f(y|x)}=\frac{f(x)}{f(y)}$. That is, for $f(x,y)$ to exist, we must be able to separate $\frac{f(x|y)}{f(y|x)}$ into a product of a function of $x$ and a function of $y$. And this is not generally true for any arbitrary pair of conditional PDF's.

But suppose our given $f(x|y)$ and $f(y|x)$ satisfies this property. From the chain rule we know $f(y) = \frac{f(y|x)}{f(x|y)} f(x)$. But $f(y)$ is the marginal PDF of $y$, so $\int_{\mathcal{Y}}f(y) = 1$. That implies

$$f(x) = \frac{1}{\int_{\mathcal{Y}}\frac{f(y|x)}{f(x|y)}}$$.

By symmetry,

$$f(y) = \frac{1}{\int_{\mathcal{X}}\frac{f(x|y)}{f(y|x)}}$$

We can show that this equation for $f(y)$ integrates to one on $\mathcal{Y}$ by writing $\frac{f(x|y)}{f(y|x)}$ in the integral in the denominator as $\frac{f(x)}{f(y)}$.

Plug them into the chain rule, we have

$$f(x,y) = \frac{f(x|y)}{\int_{\mathcal{X}}\frac{f(x|y)}{f(y|x)}}= \frac{f(y|x)}{\int_{\mathcal{Y}}\frac{f(y|x)}{f(x|y)}}.$$

Another update

It seems that this method, if it's not unfortunately flawed, can be generalized to ${x_1,x_2,...,x_n}$ for any $n$. The case where $n=2$ has been discussed above. For now, it is sufficient to consider $n=3$ to see the pattern.

Let's say we're given $f(x_1|x_2,x_3)$, $f(x_2|x_1,x_3)$, $f(x_3|x_1,x_2)$. We want to find $f(x_1,x_2,x_3)$.

Again, we use the chain rule, but more judiciously, as it involves $3!=6$ different expressions for $f(x_1,x_2,x_3)$. Even so, we still have some options to choose from. We'll look at one of them.

Consider $f(x_1,x_2,x_3)=f(x_1|x_2,x_3)f(x_2|x_3)f(x_3) = f(x_2|x_1,x_3)f(x_1|x_3)f(x_3)$. Assuming $f(x_3)\neq 0$, we can divide both sides by it and get $f(x_1|x_2,x_3)f(x_2|x_3)=f(x_2|x_1,x_3)f(x_1|x_3)$.

We'll do the same trick as in $n=2$. Rearranging the equation above, we get $f(x_2|x_3) = \frac{f(x_2|x_1,x_3)}{f(x_1|x_2,x_3)}f(x_1|x_3)$. But $\int_{\mathcal{X}_2}f(x_2|x_3)=1$, where $\mathcal{X}_2$ is the set of all possible values of $x_2$. Thus,


And we can get all other pairwise conditional PDF's similarly.

We now only need to solve for the marginal PDF's, say $f(x_3)$. But this is just a $n=2$ problem and we've solved it:

$$f(x_3) = \frac{1}{\int_{\mathcal{X}_1}\frac{f(x_1|x_3)}{f(x_3|x_1)}}$$


$$\begin{align} f(x_1,x_2,x_3) &= f(x_2|x_1,x_3)f(x_1|x_3)f(x_3) \\ &= \dfrac{f(x_2|x_1,x_3)}{\int_{\mathcal{X}_2}\dfrac{f(x_2|x_1,x_3)}{f(x_1|x_2,x_3)}\int_{\mathcal{X}_1}\dfrac{f(x_1|x_3)}{f(x_3|x_1)}}\\ &= \dfrac{f(x_2|x_1,x_3)}{\int_{\mathcal{X}_2}\dfrac{f(x_2|x_1,x_3)}{f(x_1|x_2,x_3)}\int_{\mathcal{X}_1}\dfrac{\int_{\mathcal{X}_2}\dfrac{f(x_2|x_1,x_3)}{f(x_3|x_1,x_2)}}{\int_{\mathcal{X}_2}\dfrac{f(x_2|x_1,x_3)}{f(x_1|x_2,x_3)}}} \end{align}$$

  • $\begingroup$ If this is a homework question, please explain what you have tried yourself so far and where you are stuck. Concerning the problem itself, there is no obvious way merely from the definiton of the conditional probability (assuming that you mean $P(A\cap B)$ with $P(A,B)$. It seems that the assignment makes additional assumptions (finite $\Omega$). Does it even assume classical (aka "Laplace") probabilities? $\endgroup$
    – cdalitz
    Commented Jun 19, 2022 at 7:05
  • $\begingroup$ @cdalitz It's not an assignment. I was thinking about this on myself, trying to get some deeper understanding of joint probability distributions. Is there any example where we can get two different joint probability distributions from the same pair of conditional probability distributions? $\endgroup$ Commented Jun 19, 2022 at 7:26
  • $\begingroup$ I don't get the following case: Assume two binary variables A and B that always have the same value: P(A=0|B=0) = P(B=0|A=0) = P(B=1|A=1) = P(A=1|B=1) = 1. But this holds for any value of P(A=0), which affects the joint. Thus there are multiple possible joints for the same set of conditionals. Simply put, how does knowing that A and B always have the same outcome tell me something about the base frequencies? $\endgroup$ Commented Sep 7, 2022 at 13:49


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.