So in a problem I'm working with, I'm essentially given 200 (arbitrary number) of samples from a joint distribution $P(x, y, z)$. However, what I want to be able to do is to extract the mutual information of $I(x, y | z)$, which dictates that I need the conditional distributions $P(x|z)$ and $P(y|z)$ (and if I have these, I can somewhat handle the rest. The key constraints in my problem are that

1) I do not have the closed form of the joint distribution, I know absolutely nothing about how it is distributed, only the samples

2) Whatever method I use to sample the conditionals, it has to be fast as I need to compute the mutual information at each iteration of training (i.e. can't use an MCMC method).

  • $\begingroup$ Do you know whether the marginal distributions are discrete (in which case it's relatively easy) or continuous (somewhat more difficult). $\endgroup$ – BruceET May 2 at 3:06
  • $\begingroup$ All the random variables are unfortunately continuous. $\endgroup$ – UHMWPE May 2 at 3:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.