1
$\begingroup$

Consider the task of integrating a function with respect to a multimodal distribution. Suppose I am given a set of "black box samples" from the modes of the target distribution and no other information.* For example, suppose my target distribution is a mixture of Gaussians with means $-10$ and $10$, and unit variance for both. The mixture proportion is $0.5$. A "black box sample" $X$ would be, for example, $X=(-9.1, -11, -10.3, -10.1, -9.9, 9.8, 12, 11)$. The samples are from the modes of the distribution, but not necessarily in the correct proportion.

Is there a field of research in Monte Carlo methods that sets out to correct these samples with convergence guarantees? What are some methods? Here "convergence guarantees" is to mean the estimator of the method is consistent or unbiased.

If it helps answer or simplify the question, it can be further assumed that:

  1. more samples can be drawn from this black box, and
  2. the multimodal distribution is discrete (finite support).

*That is, they're not samples from the target distribution, but just so happen to cluster around the modes. More formally, the density of the underlying distribution from which this representative set is drawn is unknown. One dreamed-up scenario is a mode-seeking algorithm that is guaranteed to sample from the exact modes of the distribution.

For example, Metropolis-Hastings can certainly benefit from such an initialization. A non-example would be importance sampling because we do not have access to the density of such a set of samples.

Analogously, parallel tempering can probability benefit as well, but not sequential Monte Carlo samplers.

In short, I have samples from the modes of a distribution (target), but they are not drawn proportionally from the target distribution. How do I correct my samples to reflect the target distribution such that sample approximations of expectations would be unbiased? (or consistent in some computational limit)

Improve this question
$\endgroup$
7
  • $\begingroup$ What is the meaning of "potentially discrete"?! $\endgroup$
    – Xi'an
    Commented Sep 1 at 14:55
  • $\begingroup$ @Xi'an updated the question: i removed that line. it was meant to express that, if it makes the problem easier, the multimodal distribution can be assumed to have a discrete domain $\endgroup$
    – fool
    Commented Sep 1 at 17:30
  • 1
    $\begingroup$ Could you elaborate on what you mean by "a set of independent but identically-distributed points in the distribution corresponding to modes of highest density/probability"? Specifically, in what sense do iid values "correspond to modes"? $\endgroup$
    – whuber ♦
    Commented Oct 7 at 14:26
  • 1
    $\begingroup$ Thank you. I still find the situation baffling, because there's insufficient information for determining anything about the rest of the distribution except that its density is lower than that near the modes. There's no possible way it could be inferred without additional information or (strong) assumptions about the whole underlying distribution. Why not describe your specific problem, then, rather than relying on the hope that your abstract description is complete and accurate? $\endgroup$
    – whuber ♦
    Commented Oct 11 at 12:42
  • 1
    $\begingroup$ @whuber Hmmm, OK I will try to reformulate the question this weekend! I appreciate your patience in guiding me on the reformulation :p! $\endgroup$
    – fool
    Commented Oct 11 at 19:57

0

Reset to default