It is well known that of all the joint distributions $p(x,y)$ with fixed marginals $p(x),p(y)$, the one with the highest entropy is: $$ p(x,y)=p(x)p(y). $$

Suppose instead that we have conditionals. Namely:

Which probability distribution $p(x,y,z)$, with fixed $p(x|y)$ and $p(x|z)$, has maximal entropy?

is there even an explicit formula?

(Note: one is tempted to take $p(x,y,z)$ such that $p(x|y,z)=p(x|y)p(x|z)$, but I think this is misleading, right?)

Feel free to edit question and tags appropriately.

  • $\begingroup$ Crossposting this to MO. $\endgroup$
    – geodude
    Jul 9 '15 at 12:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.