Learn a joint distribution from incomplete samples

Suppose I want to learn a joint distribution $p(x_1, \ldots, x_n)$ and have a collection of samples $x^k_1, \ldots, x^k_n$ for each $k$. Assume some values $x^k_i$ are unknown, so the samples are incomplete. What are some ways to learn the distribution?

I would be interested in an answer for a general distribution, but would also be interested if there is a simpler method for a Bayesian network.

• What do I know about how the unknown entries of the $k^{th}$ observation are chosen? If for example each of the $n$ entries of sample $k$ is hidden with probability $q$, then I think it should be possible to reduce the problem to ordinary density estimation.
– Dapz
Dec 12, 2012 at 20:50
• I see how it would work, but the unknown entries will be determined by human choice. Lets assume we had a complete sample, then somebody just erased some of the numbers and asked us how well we can learn from what is remaining. Dec 12, 2012 at 22:20
• Sequential imputation is one technique used in Bayesian inference with missing data. But like @Dapz says, it depends on why the data was missing(e.g. can you assume that all entries were equally likely to have come up missing? or, does your measuring instrument tend to fail when numbers are too large?) Dec 13, 2012 at 0:59