# Why doesn't P(x|y) determine P(x,y)?

This is a sort of paradox that has me confused. i just need some help clearing things up.

Let $x,y$ be two discrete, finite variables, where $x$ can take on $N$ values and $y$ can take $M$ values. Suppose someone hands you a table of the numbers $P(x|y)$. Can you determine $P(x,y)$? Intuitively, I think the answer is No. However, consider the following system of equations:

$$P(x|y)\sum_\xi P(\xi,y) = P(x,y)$$

(This is just another way to write the product rule $P(x|y)P(y)=P(x,y)$.)

Here you have $M\times N$ linear equations in the $M\times N$ unknowns $P(x,y)$. So it seems that $P(x,y)$ could be determined from this system of equations. If this is true, then knowledge of the $M\times N$ numbers $P(x|y)$ are enough to determine the $M\times N$ numbers $P(x,y)$, but this doesn't feel right. What am I doing wrong?

• You have MxN + M unknowns since you don't know P(y) (without considering constraints like all P(y) sum to 1). – George Jan 17 '16 at 13:53
• @George $P(y) = \sum_\xi P(\xi, y)$. – becko Jan 17 '16 at 13:54
• @George I realized my mistake by reading your comment. I don't know if this is what you had in mind, but I've posted an answer. Thanks! – becko Jan 17 '16 at 14:22
• Why the minus vote? Please leave a comment. – becko Jan 19 '16 at 2:40

Your $\sum_\xi P(\xi,y)$ is just a fancy way of writing $P(y)$, so your formula for $P(x,y)$ is the product of $P(x|y)$ and $P(y)$. And so indeed $P(x,y)$ was not determined just by $P(x|y)$.
• No, the system of equations I wrote only involves the unknowns $P(x,y)$, and the known values $P(x|y)$. – becko Jan 17 '16 at 13:57
• Your $P(\xi,y)$ are in the equations. That's not "Kosher" per your theory. Given that what you propose is wrong, you need to open your mind a bit to see your error. – Mark L. Stone Jan 17 '16 at 14:01
• I agree there is an error, I just don't agree that it is what you point out. If I understand correctly, what you're saying is that in my system of equations, $P(x,y)$ is determined not only by $P(x|y)$, but by both $P(x|y)$ and $P(y)$. However, I've purposefully written the equations in such a way that $P(y)$ does not appear explicitly. What I have written is a system of equations in the unknowns $P(x,y)$, where only $P(x|y)$ appears. – becko Jan 17 '16 at 14:14
The numbers $P(x|y)$ are not $M\times N$ independent numbers, because of the $N$ constrains $\sum_x P(x|y) = 1$ (one for each value of $y$). This means that in the systems of equations above you only have $M\times N - N$ independent constrains.
However, $P(x,y)$ are $M\times N - 1$ independent numbers (the $-1$ comes from the constrain $\sum_{x,y} P(x,y) = 1$). Hence you need to specify an additional $N-1$ numbers, which usually come from specifying $P(y)$.